# Prediction and Comparative Analysis of Software Reliability Model Based on NHPP and Deep Learning
## 3 Numerical Example
### Criteria  
The software reliability models utilizing NHPP and deep learning were compared using 11 criteria based on the difference between the actual and predicted values. The mean squared error (MSE) and mean absolute error (MAE) were defined as the sum of the squared distances and absolute values of the distances between the estimated and actual values, divided by the difference between the number of observations and parameters [28; 29].  
\[\text{MSE}=\frac{\sum_{i=1}^{n}(\hat{m}(t_{i})-y_{i})^{2}}{n-m},\text{MAE}= \frac{\sum_{i=1}^{n}|\hat{m}(t_{i})-y_{i}|}{n-m} \tag{13}\]  
The predictive ratio risk (PRR) was defined by dividing the distance between the actual and predicted values by the predicted value, and the predictive power (PP) was defined by dividing the distance between the actual and predicted values by the actual value [30].  
\[\text{PRR}=\sum_{i=1}^{n}\biggl{(}\frac{\hat{m}(t_{i})-y_{i}}{\hat{m}(t_{i})} \biggr{)}^{2},\text{PP}=\sum_{i=1}^{n}\biggl{(}\frac{\hat{m}(t_{i})-y_{i}}{y_ {i}}\biggr{)}^{2} \tag{14}\]  
R\({}^{2}\) is the coefficient of determination of the regression equation and determines the explanatory power by considering the number of parameters [31].  
\[\text{R}^{2}=1-\frac{\sum_{i=1}^{n}(\hat{m}(t_{i})-y_{i})^{2}}{\sum_{i=1}^{n} \left(y_{i}-\overline{y_{i}}\right)^{2}} \tag{15}\]  
The predicted relative variation (PRV) is the standard deviation of the prediction bias, where the bias is \(\sum_{i=1}^{n}\biggl{[}\frac{\hat{m}(t_{i})-y_{i}}{n}\biggr{]}\) and defined as [32]:  
\[\text{PRV}=\sqrt{\frac{\sum_{i=1}^{n}(y_{i}-\hat{m}(t_{i})-\text{Bias})^{2}}{n -1}}. \tag{16}\]  
The root mean square prediction error (RMSPE) estimates the closeness with which the model predicts the observations [32]:  
\[\text{RMSPE}=\sqrt{\text{Variance}^{2}+\text{Bias}^{2}}. \tag{17}\]  
The mean error of prediction (MEOP) sums the absolute value of the deviation between the actual data and the estimated curve and is defined as [29]:  
\[\text{MEOP}=\frac{\sum_{i=1}^{n}|\hat{m}(t_{i})-y_{i}|}{n-m+1}. \tag{18}\]The Theil statistic (TS) is the average percentage of deviation over all periods with respect to the actual values. This is defined as follows [29]:  
\[\text{TS}=100*\sqrt{\frac{\sum_{i=1}^{n}(y_{i}-\hat{m}(t_{i}))^{2}}{\sum_{i=1}^{ n}{y_{i}}^{2}}}\%. \tag{19}\]  
This criterion increases the penalty when a parameter is added to the model for a small sample [33].  
\[\text{PC}=\bigg{(}\frac{n-m}{2}\bigg{)}\text{log}\bigg{(}\frac{\sum_{i=1}^{n}( \hat{m}(t_{i})-y_{i})^{2}}{n}\bigg{)}+m\bigg{(}\frac{n-1}{n-m}\bigg{)} \tag{20}\]  
The preSSE (predicted sum squared error) refers to the sum of the differences between the predicted result value and the actual value, and the degree of estimation can be identified [34].  
\[\text{preSSE}=\sum_{i=k+1}^{n}(\hat{m}(t_{i})-y_{i})^{2} \tag{21}\]  
Based on the above criteria, we compared the software reliability model utilizing NHPP with that using deep learning. For the same comparison, the number of criterion parameters affected by the number of parameters was set to zero. The closer R\({}^{2}\) is to 1, the better the result, and the closer it is to 0 for the other 10 criteria, the better the result. The goodness of the model fit was compared by calculating the criteria using R and MATLAB R2018b.