# Bi-LSTM-Augmented Deep Neural Network for Multi-Gbps VCSEL-Based Visible Light Communication Link
## 3 Long Short-Term Memory Network  
An LSTM network, a type of recurrent neural network (RNN), is a deep learning model used for sequential data analysis [26]. LSTMs are designed to overcome the vanishing gradient problem of RNN. For RNN, the longer the data sequence, the smaller the gradient that involves the information used to update the weights and biases, which means that learning is challenging. On the other hand, in the case of LSTMs, there is a cell state vector that may prevent the gradient from vanishing. Using two unidirectional LSTM layers simultaneously to generate a Bi-LSTM network, receiving inputs forwards and backwards effectively increases the amount of information available on the network, possibly leading to better performance [27]. Figure 4 shows the structure of the Bi-LSTM and LSTM units.  
Inside the LSTM unit, there are three gates that control the flow of data to better preserve past information: forget gate \(f_{t}\), input gate \(i_{t}\) and output gate \(o_{t}\). Each gate has its own weight and bias. The symbol \(W\) and the symbol \(b\) denote weight and bias, respectively, and there are two activation functions: the sigmoid function \(\sigma\) and the hyperbolic tangent function \(tanh\). Because past values are constantly used recursively, information needs to be normalized between \(-1\) and \(1\) using a function.  
\[W=\begin{bmatrix}W_{f}\\ W_{i}\\ W_{c}\\ W_{o}\end{bmatrix},\;b=\begin{bmatrix}b_{f}\\ b_{i}\\ b_{c}\\ b_{o}\end{bmatrix} \tag{1}\]  
Figure 4: Structure of a Bi-LSTM and LSTM unit and a schematic of a neural network.  
\[\sigma(\alpha)=\frac{1}{1+e^{-\alpha}},\ tanh(\alpha)=\frac{e^{\alpha}-e^{-\alpha}}{ e^{\alpha}+e^{-\alpha}} \tag{2}\]  
The forget gate calculates the information to forget in the cell and applies the current input (\(x_{t}\)) from the current time step and the previous hidden state \(h_{t-1}\) from the previous time step to the sigmoid function to calculate the value between 0 and 1.  
\[f_{t}=\sigma\Big{(}W_{f}\cdot[x_{t},h_{t-1}]+b_{f}\Big{)} \tag{3}\]  
The input gate determines the data to be updated via the sigmoid function in a similar way to the forget gate. The candidate cell state \(\widetilde{c}_{t}\) is the calculated value between \(-1\) and 1 through the hyperbolic tangent function and is combined with \(i_{t}\) to be added to the current cell state \(c_{t}\).  
\[i_{t}=\sigma(W_{i}\cdot[x_{t},h_{t-1}]+b_{i}) \tag{4}\]  
\[\widetilde{c}_{t}=tanh(W_{c}\cdot[x_{t},h_{t-1}]+b_{c}) \tag{5}\]  
The current cell state \(c_{t}\) can be calculated by using the results of the equation above, multiplying \(f_{t}\) and the previous cell state \(c_{t-1}\) to determine the data to forget and adding the data to update. Then, the part of the cell state to be output is determined by \(o_{t}\). Based on the cell state, the parameter to export as the current hidden state \(h_{t}\) is specified. This \(h_{t}\) is again transferred to the next step and is calculated as the \(h_{t-1}\) of the next step.  
\[c_{t} =f_{t}\times c_{t-1}+i_{t}\times\widetilde{c}_{t} \tag{6}\]  
\[o_{t}=\ \sigma(W_{o}\cdot[x_{t},h_{t-1}]+b_{o}) \tag{7}\]  
\[h_{t}=o_{t}\times tanh(c_{t}) \tag{8}\]