# Deep Neural Network Equalization for Optical Short Reach Communication
## 2 Principles of Nonlinear Equalizer
### Deep Neural Network Equalizer
In order to find the weights such that the outputs \(\underline{a}^{[L]}\) are close to the target outputs \(\tilde{\underline{s}}\) and to assess performance of the DNN, a cost function \(C(\underline{s},\underline{c},\mathbf{W})\) has to be introduced. The cost function is derived using the Maximum Likelihood principle. Considering the regression problem with the training data set of \(N\) inputs \(\{\underline{s}_{1},...,\underline{s}_{N}\}\), along with the corresponding target output values \(\{\underline{\tilde{s}}_{1},...,\underline{\tilde{s}}_{N}\}\),  
\[\mathcal{D}_{\text{Dataset}}=\Big{\{}(\underline{s}_{i},\underline{\tilde{s} }_{i}):\underline{\tilde{s}}_{i}\in\mathcal{R}^{M},\underline{\tilde{s}}_{i} \in\mathcal{R}^{A},\quad i=1,...,N\Big{\}}, \tag{8}\]  
where each \(\underline{s}_{i}\) has \(M\) entries and each \(\underline{\tilde{s}}_{i}\) has \(A\) entries for all \(i=1,...,N\). The error between the DNN output and the target values is defined as  
\[\underline{\epsilon}_{i}=\underline{\tilde{s}}_{i}-\underline{a}^{[L]}( \underline{s}_{i},\mathbf{W}^{[1]},...,\mathbf{W}^{[L]}). \tag{9}\]  
Using Equation (9) and assuming a Gaussian distribution of the error with zero mean and variance \(\sigma^{2}\), as well as statistically independent symbols, the joint conditional probability density function can be derived,  
\[p(\underline{s}_{1},...,\underline{s}_{N}|\underline{s}_{1},...,\underline{s }_{N},\mathbf{W})=\prod_{i=1}^{N}\frac{1}{\sqrt{2\pi\sigma^{2}}}exp\bigg{(}-\frac {(\underline{\tilde{s}}_{i}-\underline{a}^{[L]}(\underline{s}_{i},\mathbf{W}^{[1 ]},...,\mathbf{W}^{[L]}))^{2}}{2\sigma^{2}}\bigg{)}. \tag{10}\]  
According to the ML principle, our target is to find the Maximum Likelihood estimates of \(\mathbf{W}\) for the DNN which gives the highest probability of our training data. This can be done by maximizing the joint conditional probability density function Equation (10). Figure 2:0 Figure 2:1 (12) Figure 2:2 (13) Figure 2:3  
By applying the logarithm on Equation (12), we obtain the quadratic cost function. Thus, minimizing the quadratic cost function corresponds to finding the ML estimate of weights for the DNN. Equation (14) defines the final overall cost of all samples in the dataset. In order to find the weights, which minimize the quadratic cost function, the gradient descent algorithm [15] is used. Here, the gradients of the cost function have to be computed with respect to the DNN parameters. An efficient method is the common backpropagation algorithm [16]. The backpropagation algorithm approaches the derivatives of the cost function from the training set with respect to the network parameters. It applies the chain rule for derivatives very efficiently. The derivation of the cost function with respect to the weights in the \(L\)th-layer down to the first layer by applying the chain rule, can be expressed as follows  
Figure 2:4  
Figure 2:5  
Figure 2:6  
where \(\sigma^{\prime}(\cdot)\) denotes the derivation of the activation function in the current layer. Equation (18) is executed iteratively for subsequent layers. The final gradients are used to update the weights by shifting the old weight values towards negative direction of the gradient, given by the Equation (19). The parameter \(\alpha\) denotes the learning rate and is adaptively adjusted by the adaptive moment estimation (ADAM) [15] in order to enable the gradient algorithm to move gently towards the global minimum. Figure 2:7