# Blind Recognition of Forward Error Correction Codes Based on Recurrent Neural Network
## 3 The Proposed Approach
### Proposed Deep Neural Network Methodology  
#### 3.2.1 Introduction of Simple RNN  
In recent years, neural network is a hot research object. Further research on artificial neural network promotes the development of deep learning. At present, deep learning has made a series of remarkable achievements in image, text, video, Automatic Speech Recognition (ASR) and other application fields. RNN is a typical network structure in the field of deep learning. Its characteristic is that it can model the temporal changes of sequence data and is suitable for processing it. Because of this characteristic, RNN has been successfully applied in language modeling, speech recognition, sentiment analysis, machine translation and many other aspects and is expected to show a better application prospect in these aspects. Compared with CNN, the innovation of RNN lies in that by introducing the processing of the state of the last time step, the network can make full use of the information of the previous state in the current time and can model the change of sequence data in time, which is suitable for processing the data that changes in time dimension. RNN includes input layer, hidden layer and output layer. In addition to fully connection between layers, the nodes of the hidden layer are connected to each other. However, there is no correlation between the nodes of each layer. Through such a design, the state of the hidden layer at the last moment can be added to the calculation of the hidden layer and the output layer at the current time, which is equivalent to using the previous state information to a certain extent. Theoretically, speaking, the current input of the hidden layer can not only be limited to the output of the previous stage of it, but also be related to the hidden layer outputs of the previous several time slices. However, in practice, in order to reduce the complexity of network structure, it is usually assumed that the output of the current hidden layer is only related to the output of the previous stage. Simple RNN is a typical structure and Figure 2 shows the basic model of it. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**FEC Codes** & **Code Rates** & \(n\) & **k** & **Generators** \\ \hline Convolutional & 1/2 & 1 & 2 & [23; 35] \\  & & & & [53; 75] \\  & & & & [133; 171] \\  & & & & [237; 345] \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Convolutional codes parameter descriptions. As the figure shows, the hidden layer of Simple RNN is the key of its structural design. In addition, to passing from the input layer to the hidden layer and from the hidden layer to the output layer, the hidden layer is also related. For Simple RNN, the forward propagation process is as follows:  
\[a_{h}^{t}=\sum_{i=1}^{I}w_{ih}x_{i}^{t}+\sum_{i=1}^{I}w_{h^{t}h}b_{h^{\prime}}^{t -1} \tag{4}\]  
\[b_{h}^{t}=\theta_{h}(a_{h}^{t}) \tag{5}\]  
\[a_{h}^{t}=\sum_{i=1}^{H}w_{hk}b_{h}^{t} \tag{6}\]  
\(a_{h}^{t}\) represents the calculation output of the hidden layer and \(a_{h}^{t}\) represents the output of the output layer. As the Equation (4) shows, the calculation of \(a_{h}^{t}\) is divided into two parts. The first part is both connected to the input layer and calculated. The second part is the self-connecting of the hidden layer, that is, the output of the hidden layer from the last time step after passing the activation function, which \(w_{h^{\prime}h}\) represents the weight parameters between the self-connecting of the hidden layer. \(\theta_{h}\) represents the activation function through which the hidden layer output passes. The training optimization of Simple RNN is also optimized by BP algorithm. However, Simple RNN is propagating forward, it takes advantage of the timeline information. Therefore, the back propagation algorithm of Simple RNN transfers residuals back to the last time from the next time and the time dimension is equivalent to the number of layers of the deep neural network.