# Deep Neural Network Equalization for Optical Short Reach Communication
## 2 Principles of Nonlinear Equalizer
### Deep Neural Network Equalizer  
Along with other science and engineering disciplines, the optical communication community is adopting ML for a broad range of problems [11; 12; 13]. Neural network (NN) is a subcategory of ML, which automatically learns systematic features within an arbitrary data set. It then can extract and extrapolate these features to new data. NNs computing structures are built from several layers of artificial neurons. A single artificial neuron is a processing unit with a number of inputs and one output. Each input is associated with a weight. The neuron firstly computes an activation by summing up the particular weighted inputs  
Figure 1: General nonlinear volterra equalizer. Figure 2: Memory polynomial nonlinear equalizer. and a bias term. Secondly, an activation function \(\sigma(\cdot)\) is applied to obtain the neurons's output \(a=\sigma(z)\). The neurons's behaviour is defined by the weights, bias and the activation function [14]. Interconnecting multiple neurons builds a NN. If more than one hidden layer is used, the NN is called a deep NN, referred to as DNN. DNNs are convenient to model unknown highly complex relations. As such, they are capable of learning nonlinear equalization. In machine learning terms, nonlinear equalization can be treated as a problem of supervised nonlinear regression. A regression DNN learns a function \(f(\cdot):\mathcal{R}^{M}\rightarrow\mathcal{R}^{A}\) by training on a dataset, where \(M\) is the input dimension and \(A\) is the output dimension. Figure 3 shows a basic architecture of a regressive neural network with multiple hidden layers. The bias term is included by an additional branch to each neuron. The notation is as follows [14] :  
* \(\underline{s}=[s_{1},...,s_{M}]\) denotes the input signal,
* \(\underline{s}=[s_{1},...,s_{A}]\) denotes the equalized signal,
* \(\boldsymbol{W}^{[l]}\) denotes a weight matrix (including the bias) between the \((l-1)\)-th layer to the \(l\)-th layer,
* \(a_{i}^{[l]}\) denotes the output of the \(i\)-th neuron in the \(l\)-th layer,
* \(z_{i}^{[l]}\) denotes the summed up input to the \(i\)-th neuron in the \(l\)-th layer,
* The activation function \(\sigma_{i}^{[l]}\) is the relation between \(z_{i}^{[l]}\) and \(a_{i}^{[l]}\) of the \(i\)-th neuron in the \(l\)-th layer,  
where \(l\) denotes the current layer and \(H\) the number of nodes in the hidden layers. Using these notations the output can be expressed as follows [14],  
\[\underline{a}^{[0]}=\underline{s}, \tag{4}\]  
\[\underline{z}^{[l]}=\boldsymbol{W}^{[l]}\underline{a}^{[l-1]}, \tag{5}\]  
\[\underline{a}^{[l]}=\sigma^{[l]}\big{(}\underline{z}^{[l]}\big{)}, \tag{6}\]  
where the first line, Equation (4), denotes the input layer and the second as well as the third lines, Equations (5) and (6), are executed iteratively for subsequent layers to obtain the output \(\underline{a}^{[L]}\). For regression problems, the activation function in the last layer may differ from the activation functions used in the hidden layers. The hidden layers usually use nonlinear functions, while the type of activation function at the last layer should be matched to the application. This output activation function has to represent the overall dynamic range of the target signal. A common choice is therefore the identity activation function for a regression problem. The identity activation function is applied to each neuron at the output layer. The DNN models the relation of \(\underline{s}\) and \(\underline{s}\) as being an output of a DNN with respect to the input \(\underline{s}\) and the nonlinear functions. The equalized signal is obtained at the output of the DNN and can be expressed as follows,  
\[\underline{s}_{\text{DNN-NLE}}=\underline{a}^{[L]}(\underline{s},\boldsymbol{ W}). \tag{7}\]  
Figure 3: \(L\)-layers feed-forward deep neural networks (DNN) with \(M\) nodes in the input layer, \(H\) nodes in the hidden layers and \(A\) nodes in the output layer.