# Machine Learning Applications for Short Reach Optical Communication
## 3 Machine Learning for Short-Reach Optical Communication System Signal Processing
### Machine-Learning-Based Equalizer for Short-Reach Optical Communications
#### 3.1.1 Feed Forward Neural Network Based Equalizer
Hence, the capability of FFNN in the PAM-based high-speed short-reach optical communication system has also been studied [103], where 32 GBaud/s PAM-8 signal generated by an integrated electro-absorption modulated laser and transmitted over 4 km single mode fiber (SMF) has been equalizer. The structures of the neural network-based equalizer is also an FFNN similar to Figure 8. The normalized least-mean-square (NLMS) algorithm minimizes the MSE of the training symbols. In this work, an FFNN with 9 input nodes is used. Results show a better equalization performance (50% lower BER at 2 dBm output power) is achieved compared with the 21-tap conventional symbol space linear FFE. However, this work assumes the channel is static, and the weights of the neural network and input layer time delay are also kept static. Thus, in reality, this FFNN model cannot adapt to possible channel changes over time. Whilst the capability of FFNN as the signal equalizer has been demonstrated, the complexity of the neural network is an essential practical hurdle that needs to be considered. This is particularly critical in short-reach optical communications due to the more stringent complexity requirement than long-haul transmissions. To reduce the complexity, a compressed neural network equalizer using the iterative pruning approach has been proposed and demonstrated [104]. The pruning procedure is shown in Figure 9, and it reduces the network complexity, whereas the retraining procedure restores performance lost as a result of the pruning. This process of pruning and retraining is conducted recursively, and a sparse neural network structure with few connections can be generated after a number of iterations. The compressed neural network is used as the equalizer in a VCSEL-based short-range system with 100 m MMF, and up to 112 Gbps PAM-4 or PAM-8 signals are transmitted. In the compressed neural network, the input and hidden layers have 51 nodes, and the  
Figure 9: Iterative pruning process of the compressed FFNN [104]. output layer scale is decided by the modulation format, where 4 and 8 output nodes are used for PAM-4 and PAM-8 formats, respectively. The cross-entropy loss function was used to train the neural network to improve the classification performance. Rectified Linear Unit (ReLU), a prominent nonlinear activation function, was chosen as the activation function of a single hidden layer. Results reveal that, using the iterative pruning technique instead of fully connected FFNN, can result in a connection decrease of up to 71% at a BER threshold of \(3.8\times 10^{-3}\)[104]. When 2000 connections of the neural network are pruned, the BER of the one-time pruning and iterative pruning algorithms (\(2\times 10^{-4}\)) is lower than that of a fully connected FFNN (\(1\times 10^{-3}\)) in a PAM-8 back-to-back system. This is because that the network's redundancy is reduced, and hence the network's generalization ability is improved.