## 4 Numerical Examples
### Model Comparison Criteria  
In this paper, the performance of models is evaluated using six model comparison criteria:  
1. Mean Square Error (\(MSE\)) [55]. This metric is used to assess how well software RMs fit and predict performance. It calculates the deviation between the estimated fault number of a software RM and the actual fault number detected during software testing. \[MSE=\frac{1}{n}\sum\limits_{k=1}^{n}{(\psi(t_{k})-\Lambda(t_{k}))^{2}}\] (7) and \[MSE_{predict}=\frac{1}{m}\sum\limits_{k=n-m+1}^{n}{(\psi(t_{k})-\Lambda(t_{k}) )^{2}}\] (8)
2. R-square (\(R^{2}\)) [55]. This criterion is used to evaluate the fitting performance of software RMs. \[R^{2}=1-\frac{\sum\limits_{k=1}^{n}{(\psi(t_{k})-\Lambda(t_{k}))^{2}}}{\sum \limits_{k=1}^{n}{(\Lambda(t_{k})-\sum\limits_{k=1}^{n}{\Lambda(t_{k})/n})}^{2}}\] (9)  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Model Name** & **Mean Value Function (MVF)** & **Model Description** \\ \hline
1 & G-O model [47] & \(\psi(t)=a(1-\exp(-bt))\) & CSS RM \\ \hline
2 & Delayed S-shaped model (DSS) [50] & \(\psi(t)=a(1-(1+bt)\exp(-bt))\) & CSS RM \\ \hline
3 & Inflection S-shaped model (ISS) [51] & \(\psi(t)=\frac{a(1-\exp(-bt))}{1+\exp(-bt)}\) & CSS RM \\ \hline
4 & Yamada Imperfect-2 model [52] & \(\psi(t)=a(1-\exp(-bt))(1-a\;/\;b)+\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\; \;\;\;\;\;\;\;\  
3. Root Mean Square Error (_RMSE_) [55]. This measures the square root of the distance between the estimated values and the actual observations. In general, it is used to evaluate the fitting and predictive performance of software RMs. \[RMSE=\sqrt{\frac{\sum\limits_{k=1}^{n}{{{(\psi(t_{k})-\Lambda(t_{k}))}^{2}}}}{n}}\] (10) and \[RMSE_{predict}=\sqrt{\frac{\sum\limits_{k=n-m+1}^{n}{{{(\psi(t_{k})-\Lambda(t_{ k}))}^{2}}}}{m}}\] (11)
4. Kolmogorov-Smirnov test (K-S test) [56]. This metric is intended to assess how well software RMs fit. At every point, it calculates the absolute deviation between the expected distribution function from the model and the normalized cumulative distributions of the actual observed rates. \[D_{i}=\sup_{y}{\mid F_{i}(y)-F(y)\mid}\] (12) Equation (12) is also called Kolmogorov distance (_KD_). 5. The The Theil statistic (_TS_) [55]. This is the average distance percentage between the estimated values from the model and the actual values. \[TS=\sqrt{\frac{\sum\limits_{k=1}^{n}{{{(\psi(t_{k})-\Lambda(t_{k}))}^{2}}}}{ \sum\limits_{k=1}^{n}{{\Lambda(t_{k})}^{2}}}}\times 100\%\] (13) and \[TS_{predict}=\sqrt{\frac{\sum\limits_{k=n-m+1}^{n}{{{(\psi(t_{k})-\Lambda(t_{ k}))}^{2}}}}{\sum\limits_{k=n-m+1}^{n}{{{\Lambda(t_{k})}^{2}}}}}\times 100\%\] (14)
6. _Bias_ [55]. This is the sum of the deviation between the observed values and the estimated values from the model. \[Bias=\frac{\sum\limits_{k=1}^{n}{{{(\psi(t_{k})-\Lambda(t_{k}))}}}}{n}\] (15) and \[Bias_{predict}=\frac{\sum\limits_{k=n-m+1}^{n}{{{(\psi(t_{k})-\Lambda(t_{k}))}} }}{m}\] (16)  
In Equations (7)-(16), \(\psi(t_{k})\) denotes the estimated number of faults detected by time \(t_{k}\). \(\Lambda(t_{k})\) represents the number of faults observed by time \(t_{k}\). \(n\) and \(m\) are the sample size of the fault data. In Equations (8), (11), (14), and (16), the first (n-m) fault times are used to estimate the model parameter values, and the remaining fault times are used to compute the prediction values. Note that the smaller the _MSE_ (_MSEpredict_), D\({}_{\mathbf{k}}\), _RMSE_ (_RMSEpredict_), _TS_ (_TSpredict_), and _Bias_ (_Biaspredict_) values are, the better the predictive or fitting performance of the model is. The larger the value of \(R^{2}\) is, the better the model's fitting ability is.