## 4 Results and Comparison
### RUL Estimation for Test Instance Bearings
#### 4.1.5 Summary of Results and Discussions  
In the IEEE PHM 2012 Prognostic Challenge, a scoring function was used to evaluate the RUL estimation results as follows [15]. Based on the relative error \(Er\) obtained using Equation (17), the score of RUL estimation for each test bearing is defined as:  
\[A=\left\{\begin{array}{rl}exp^{-ln(0.5)\cdot(100\cdot Er/5)}\ if\ Er\leq 0\\ exp^{+ln(0.5)\cdot(100\cdot Er/20)}\ if\ Er>0\end{array}\right. \tag{18}\]  
This scoring function can penalize both underestimates and overestimates such that **(i)** the perfect estimate corresponding to a perfect score of 1 (i.e., \(Er=0\)), (ii) score deduction is given to estimates that are shorter than the actual bearing RUL (i.e., \(Er>0\)), and (iii) a more severe score deduction is given to estimates that exceed the actual bearing RUL (i.e., \(Er<0\)). Figure 15 depicts the scoring function. The overall estimation score averages the scores for \(N\) test bearings.  
\[Score=\frac{1}{N}\sum_{i=1}^{N}A_{i} \tag{19}\]  
The RUL estimation results for Bearing1-3, 4, 5, 6, and 7 (under the operating condition I) are summarized in Table 6, which achieve the relative errors \(-6.58\%\), \(32.4\%\), \(19.6\%\), \(69.2\%\), and \(-3.15\%\), respectively. For Bearing1-3, Bearing1-5, Bearing1-6, and Bearing1-7, the similarity evaluation was made using the proposed I  
Figure 14: Similarity evaluation for RUL estimation of Bearing1-7 using Bearing1-2.  
Figure 15: Scoring function for RUL estimation.  
augmentation method. For Bearing1-4, its RUL was estimated directly using the reference of the close-to-failure stage duration of Bearing1-2, because the test bearing is considered to have entered the close-to-failure stage at its last time cycle. In this study, we used (\(d_{min}<0.5000\)) as the criteria for identifying a pair of similar feature sequences between the test and learning instance.  
In this paper, the seven synthetic feature sequences associated with different scaling factors are selected to calculate the final RUL using the ensemble method, that is, \(M=7\) in the Equation (16). Worthy to be studied, we use the Mann-Withney test [24,25] to investigate the statistical difference among the prediction results obtained by setting \(M\) to 1, 3, 5, 7, and 9. The hypotheses include: (1) H0-the medians of two sets of RUL prediction scores (e.g., \(M=3\) vs. \(M=1\)) are the same, and (2) H1-the medians are the same for the two sets of scores [25]. A \(p\)-value of 0.05 is used for the hypothesis test. Table 7 shows the prediction scores for test instances Bearing1-3, 1-5, 1-6, and 1-7 obtained with five different settings, \(M=1\), 3, 5, 7, and 9. Note that Bearing1-4 is not included in the test, as the ensemble method is not applied for this instance. The overall score (i.e., average score) of \(M=7\) for the four test instances apparently outperforms others in the comparison. However, we cannot conclude that there are significant differences among the different settings, as all \(p\)-values are greater than 0.05 by the Mann-Withney test. Reasonably we consider that the very limited number of test instances might have affected the test of significant difference in statistics.