# Statistical Power Analysis in Reliability Demonstration Testing: The Probability of Test Success
## 2 Probability of Test Success
As the hypothesis of reliability demonstration tests stay the same, as defined in (1) and (2), the statistical power of a reliability demonstration test is called the _Probability of Test Success_, \(P_{\rm R}\). Therefore, merely considering a confidence level \((1-\alpha)\) in reliability test planning is not sufficient, as the analysis of the applicability of the test \((1-\beta)\) to the individual scenario is neglected [38; 39; 40; 45; 46; 47]. Because the power of a test, and therefore the Probability of Test Success, has to be calculated for a specific effect size [35], and because the probability of detecting an effect changes with a change in the effect size, a measure of the effect size has to be established for reliability tests. For this purpose, the safety distance,\(s\), of the product is used, as introduced by [PERSON] et al. [46]. The safety distance describes the distance of the required lifetime, \(t_{\rm r}\), to the actual lifetime of the product, \(t_{\rm p}\), in accordance with prior knowledge about the failure distribution, and is defined as follows:  
\[s\equiv 1-\frac{t_{\rm r}}{t_{\rm p}} \tag{3}\]  
where \(t_{\rm p}\) is the lifetime with the required reliability in accordance with the prior knowledge and \(t_{\rm r}\) is the required lifetime at the required reliability. The safety distance, \(s\), is equal to zero if the prior knowledge states that the required lifetime is equal to the actual service life of the product. Using this metric, the effect size, \(\Delta\), can be formulated as follows:  
\[\Delta\equiv t_{\rm p}-t_{\rm r}=s\cdot t_{\rm p} \tag{4}\]  
In order to calculate the confidence level, \(C\), as well as the \(P_{\rm ts}\), the distributions of the test statistic under validity of the null and the alternative hypothesis must be obtained. Using the safety distance from (3) as well as the effect size from (4), the test statistic, \(\tau\), of reliability demonstration tests is defined as  
\[\tau\equiv t_{R_{\rm r}}-t_{\rm r}. \tag{5}\]  
This is because it measures the distance from the lifetime quantile of the test to the expectation of it under validity of the null hypothesis (\(\tau=0\) for \(t_{R_{\rm r}}=t_{\rm r}\)) and the prior knowledge about the failure distribution. The null distribution \(f_{H_{0}}(\tau)\) and alternative distribution \(f_{H_{1}}(\tau)\) of the test statistic are defined by the prior knowledge of the failure distribution and the sampling scheme of the test. Regarding this approach, it can be concluded that prior knowledge of the failure distribution is indispensable for statistically profound planning of reliability tests, as the actual effect size of the product is determined by this information. The distribution of the test statistic under validity of the alternative hypothesis (alternative distribution \(f_{H_{1}}(\tau)\)) can be obtained by shifting the confidence distribution [48] of the calculated test statistic under \(H_{1}\) by the required lifetime, \(t_{\rm r}\). The confidence distribution is dependent on the sample size and the test to be analyzed as well as on the failure distribution, and can be calculated by means of an MCS and a maximum likelihood estimation (MLE), for example. The distribution of the test statistic under validity of the null hypothesis (null distribution \(f_{H_{0}}(\tau)\)) can be obtained through the same procedure, although in contrast with the alternative distribution it must be calculated via the failure distribution, which is valid under \(H_{0}\); therefore, \(s=0\) (effect size of zero, \(\Delta=0\)). Where a Weibull distribution is concerned, the shape parameter should be kept the same in order to maintain the characteristic of the failure mode at hand (see [10]), whereas the scale parameter should be adjusted accordingly. Taking the null distribution \(f_{H_{0}}(\tau)\), the significant effect size, \(\Delta_{\rm crit}\), can be calculated in accordance with the required confidence, \(C_{\rm r}\).