# Prediction and Comparative Analysis of Software Reliability Model Based on NHPP and Deep Learning
## 2 Software Reliability Model
### Software Reliability Model Using Deep Learning
#### 2.2.2 Recurrent Neural Network  
The recurrent neural network is a model specialized for sequential data structures. It has cells that can remember the information of a previous point in the hidden layer of the most basic deep neural network. Therefore, it is suitable for time-series data because it is \begin{table}
\begin{tabular}{c c c c} \hline
**No.** & **Model** & **Mean Value Function** & **Note** \\ \hline
10 & [PERSON] et al. (UDPF) [16] & \(m(t)=\mathrm{N}\Big{[}1-\Big{(}\frac{\beta}{\alpha+b\mathrm{f}-\ln(br+1)}\Big{)} ^{\alpha}\Big{]}\) & S-shape \\  & & Dependent \\ \hline \end{tabular}
\end{table} 
Table 1: _Cont._designed to remember information from the past and influence future events. Figure 1a shows an image of the RNN model, and the formula is shown in Equation (10) [24]. \[h_{t}=tanh(W_{h}h_{t-1}+W_{x}h_{t}+b) \tag{10}\]  
Its network structure reflects past time; however, it has a vanishing gradient problem that prevents it from accurately reflecting the information about the time that existed a long time ago as the time point becomes longer [25]. To address this problem, LSTM, which has a structure that can learn long-term dependence, was built. It contains the same hidden layer as the RNN structure; however, its internal structure differs. In LSTM, a memory cell c exists. This cell has a forget gate that determines how much previous information is reflected, an input gate that includes new information g, which indicates how much new information is reflected, and an output gate that indicates how much previous information is transferred to the next layer. The activation functions used at this time were the sigmoid function and the hyperbolic tangent. Information about the past is delivered by passing through three gates and memory cells. Through this, the gradient loss problem can be solved. Each gate follows Equation (11). Figure 1b shows the LSTM model [26]. \[ \begin{split}&\text{f}_{t}=\sigma(x_{t}W_{x,t}+h_{t-1}W_{h,t}+b_{t}) \\ & g_{t}=\text{tanh}\Big{(}x_{t}W_{x,g}+h_{t-1}W_{h,g}+b_{g}\Big{)} \\ & i_{t}=\sigma(x_{t}W_{x,i}+h_{t-1}W_{x,i}+b_{i})\\ & o_{t}=\sigma(x_{t}W_{x,o}+h_{t-1}W_{x,o}+b_{o})\\ &\qquad c_{t}=\text{f}_{t}\circ c_{t-1}+g_{t}\circ i_{t}\\ &\qquad h_{t}=o_{t}\circ\text{tanh}(c_{t})\end{split}  \tag{11}\]  
LSTM can solve the gradient loss problem in RNN; however, when going to the next step, LSTM goes through four operation processes if only one operation is performed. Processing big data may be inefficient because the number of computations increases fourfold. Complementing this, the gated recurrent unit (GRU) transmits a value to the next layer through three calculation processes in one layer. The GRU integrates the forget and input gates in the LSTM to create an update gate. In addition, it replaces the output gate with a reset gate, which determines the amount of previous information to be forgotten. The update gate determines how long the previous information should be kept. Through  
Figure 1: (**a**) Structure of RNN model; (**b**) Structure of LSTM model; (**c**) Structure of GRU model. Equation (12), the network learns while transferring to the next layer. Figure 1c shows an image of the GRU model [27]. \[ \begin{split} r_{t}&=\sigma(x_{t}W_{x,r}+h_{t-1}W_{h,r }+b_{r})\\ z_{t}&=\sigma(x_{t}W_{x,z}+h_{t-1}W_{h,z}+h_{z})\\ g_{t}&=tanh\Big{(}x_{t}W_{x,g}+(r_{t}\circ h_{t-1})W_ {h,g}+b_{g}\Big{)}\\ h_{t}&=(1-z_{t})\circ g_{t}+z_{t}\circ h_{t-1}\end{split}  \tag{12}\]  
The software reliability model proposed in this study adds a deep neural network layer to add training depth to the predicted model, which reflects the past viewpoint through the hidden layer of the recurrent neural network. It proposes a learning model that goes through a deeper hidden layer to reflect the results of the past rather than just reflecting the past time of RNN series. This allows us to propose a software reliability model that can be generalized by solving problems that are only well suited to special cases of previously developed software reliability models.