# Statistical Power Analysis in Reliability Demonstration Testing: The Probability of Test Success
## 4 Comparison of the Calculation Methods for the Probability of Test Success
### Comparison Using Failure-Based Tests  
For EoL tests, the following methods are used to calculate \(P_{\text{ls}}\):  
* General method (General);
* Approximate method (Approximate);
* Test simulation method (Test sim.). In Figure 5, \(P_{\text{ls}}\) is calculated using the approximate method as well as the general method for a parameter-free sample quantile estimation (sample q.) and a quantile estimation via distribution estimation using MLE. It can be seen that the general method and the approximate method (see (27)) show very good agreement if parameter-free quantile estimation is used. On the other hand, the approximation based on the MLE (see (41)) shows very good agreement with the general method if the MLE is used for quantile estimation. Therefore, the approximation solely based on the sample quantiles (27) is to be used if empirical sample quantile estimation is used in the test. If the very common MLE is used for distribution parameter and quantile estimation, the approximate approach in (41) should be used instead of (27). The offset between the \(P_{\text{ls}}\) of the sample quantile estimation and the MLE is due to the nature of the estimation method. The MLE uses the information of all failure times, whereas the sample quantile estimation primarily uses information about the number of failures. In Figures 6-10, the \(P_{\text{ls}}\) of the general method, the approximate method, and the test simulation method is shown for the variables \(n\), \(s\), \(R_{\text{r}}\), \(C_{\text{r}}\), and \(b\), respectively. Because MLE  
Figure 4: \(P_{\text{ls}}\) of the SR test for \(R_{\text{r}}=C_{\text{r}}=90\%\), \(t_{\text{r}}=0.2\) (\(s=0.577\)) and Weibull distributed failure times with \(b=3\), \(T=1\). Calculated using the general method with different MCS iteration numbers and the exact method. is used, Equation (41) is used for the approximate method. For each calculation, 100,000 iterations were performed for the bootstrap of the general method and the MCS of the test simulation method. The approximate method shows a very good fit to the values of the general calculation method. However, the test simulation method using a Fisher confidence bound shows a significant offset in respect of the other methods. In particular, with sample size \(n\) (see Figure 6), an increase in \(P_{\mathrm{ls}}\) with smaller sample sizes for \(n<20\) can be seen. This is due to the enhanced effect of the MLE bias, which causes the estimates of the Weibull parameters to be noticeably biased in small samples [45]. However, for very large sample sizes the test simulation method shows identical values of \(P_{\mathrm{ls}}\) compared to the general method for \(n>100\). This seems to be a large enough sample size to remedy the bias. Additionally, these matching values verify the approach following the hypothesis testing framework and its calculation according to the general method and the approximate method. Looking at the values of \(P_{\mathrm{ls}}\) with regard to the safety distance \(s\) in Figure 7, a similar behavior can be seen. The general method shows conformity with the test simulation method for large  
Figure 5: \(P_{\mathrm{ls}}\) of the EoL test for \(R_{\mathrm{r}}=C_{\mathrm{r}}=90\%\), \(s=0.1\), and prior knowledge of \(b=3\). Calculated for sample quantile estimation (sample q\({}_{\mathrm{r}}\)) and MLE quantile estimation (MLE). Although the sample size is to be an integer, the curves are interpolated to obtain a better understanding of the trajectory. Figure 6: \(P_{\mathrm{ls}}\) of the uncensored EoL test for \(R_{\mathrm{r}}=C_{\mathrm{r}}=90\%\), \(s=0.1\) and prior knowledge of \(b=3\). All three methods use the MLE. Although the sample size is to be an integer, the curves are interpolated to obtain a better understanding of the trajectory. For a censored EoL test, the approximate method remains a very good approximation. Figure 11 shows \(P_{\mathrm{ts}}\) with regard to sample size of the different methods for moderate right-censoring of \(30\,\%\) of the specimen.