# Blind Recognition of Forward Error Correction Codes Based on Recurrent Neural Network
## 3 The Proposed Approach
### Proposed Deep Neural Network Methodology
#### 3.2.2 Weight Initialization of Simple RNN  
Parameter initialization is the key part of neural networks. Excellent initialization can not only greatly improve the training efficiency, but also greatly improve the final training effect. At present, the weight initialization methods commonly used in neural networks are as follows:  
1. Gauss initialization method initializes the weight to a random number that conforms to a Gaussian distribution with a mean of 0 and a standard deviation of 0.01. This is by far the most widely used initialization method in deep learning. 2. Xavier initialization method is to randomly initialize weights from a Gaussian or standard distribution multiplied by a certain ratio. Figure 2: The basic model of simple Recurrent Neural Network. 3. MSRA initialization method is to initialize to a Gaussian distribution with a mean of 0 and a variance of \(\frac{2}{n}\). In this section, we devote to solve the gradient explosion and gradient convergence problems without deeply changing the network structure of Simple RNN through weight initialization work and at the same time maintain long-term memory and have the ability to learn long time series. In this paper, a RNN weight initialization method proposed by [PERSON] et al. (2018) is adopted to improve the performance of Simple RNN. Gaussian initialization is adopted in this algorithm. For the self-connect weight of the hidden layer, initialize it here with the matrix \(W\) shown below. \[ \begin{array}{ccccc}1+\theta&\theta&\dots&\dots&\theta\\ \theta&1+\theta&\theta&\dots&\theta\\ \theta&\dots&1+\theta&\dots&\theta\\ \theta&\dots&\dots&1+\theta&\theta\\ \theta&\dots&\dots&\theta&1+\theta\end{array}  \tag{7}\]  
As shown in Equation (7), the matrix used can be regarded as adding an identity matrix on the basis of a Gaussian initialization:  
\[W=I+A \tag{8}\]  
\(I\) is the identity matrix, \(A\) is the Gaussian distribution matrix. \(\theta\) is a Gaussian distribution with a mean of 0 and a variance of 0.01. The above initialization method for Simple RNN is shown in the Figure 3.  
As shown in the Figure 3, \(X\) represents the input of Simple RNN at different time steps, \(H\) represents the state of the hidden layer at different time steps and \(O\) represents the state of the output layer of network at different time steps. In the ordinary neural network, the parameters are different between each layer. The parameters of each layer are the same for different time steps, since they are unrolled along the timeline. Therefore, for different time steps, the weight from the input layer to the hidden layer obtains \(W_{1}\), the weight from the hidden layer to the output layer is always \(W_{2}\) and the weight of the self-connecting of the hidden layer is always \(W_{3}\). The weight of \(W_{3}\) is initialized to \(I+A\). \(W_{1}\) and \(W_{2}\) are  
Figure 3: The weight initialization of Simple RNN. initialized with the Gaussian distribution mentioned above. According to Equation (4), when the residual is calculated and it returned from the hidden layer is received by a time step, the residual of the next time will be multiplied by the weight of the hidden layer. This internal weights of hidden layer neurons are initialized as the matrix \(I+A\) which is described above, on the premise of assuming no other residual accumulation, as the hidden layer of residual travels forward by back propagation algorithm and is multiplied by unit matrix \(I\), the next time the hidden layer of residual will remain the same and is multiplied by matrix \(A\) which is in line with gaussian distribution. The residual is multiplied by a certain coefficient to control the influence of the previous time step state on this time step. To a certain extent, the improved Simple RNN has the ability to learn long time series correlation, which can have better performance. Compared with the original, the improved Simple RNN has enhanced performance, effectively suppress the gradient disappearance, reduce the computational complexity and improve the training efficiency.