# Rail Diagnostics Based on Ultrasonic Guided Waves: An Overview
## 9 Performance Analysis
### On-Board Passive System: Performance Analysis  
Some DI traces obtained for the on-board passive defect detection system described in Section 5.2 are illustrated in Figure 24 (its plots have been generated generated on the basis  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Test Speed** (**km/h**)} & \multicolumn{4}{c}{**PD (\%) Achievable for Specific PFAs**} & \multirow{2}{*}{**AUC (\%)**} \\ \cline{2-2} \cline{4-6}  & **0\% PFA** & **1\% PFA** & **5\% PFA** & **10\% PFA** \\ \hline
1.6 & 65.38 & 76.92 & 86.54 & 92.31 & 97.58 \\
8 & 7.08 & 47.79 & 78.76 & 85.84 & 95.03 \\
16 & 0.00 & 29.23 & 47.69 & 56.92 & 77.90 \\
24 & 1.92 & 9.62 & 28.85 & 32.69 & 69.05 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Significant PD and PFA values extracted from the cumulative ROC curves of Figure 23 [47].  
Figure 23: Cumulative ROC curves for runs executed at four different speeds: (**a**) 1.6 km/h; (**b**) 8 km/h; (**c**) 16 km/h; (**d**) 24 km/h. The results associated with three different values of the detection threshold are identified by specific labels.  
of the data available in Ref. [49]). Additional results illustrated in Ref. [66] have evidenced that:  
1. The zones in which the received signal was strong enough for defect detection were primarily localized in sections of curved tracks, where wheels were flanging, so generating stronger excitation signals.
2. As speed decreases, a larger portion of the run becomes sub-optimal, since the presence of a defect cannot be easily detected.
3. At high speeds (say, higher than 96 km/h), an optimal source excitation energy is generated; this allows for achieving a stable Green's function.
4. Poor results were found in areas where the source excitation energy was too low to be detected by the receivers, e.g., in most of the tangent portion of the track. affecting the performance of the defect detection algorithms (and, in particular, of the ICA; see Section 7.4).  
Figure 24: Damage index traces referring to the RDTF test track at 25 mph (40 km/h) in three selected test zones.  
The performed tests did not allow for assessing the true repeatability of the achieved results; for this reason, further tests have been planned to investigate the reliability of the considered system [49]. Moreover, it has been found that, as long as acoustic excitation is detectable, the strength of the received signal does not play an important role in the reconstruction stability. However, instability is found when the signal intensity is comparable to the noise floor of the receiver sensors.  
Based on the considerations illustrated in Section 5.2.4, concerning the conflicting needs of selecting a proper test speed and achieving an accurate localization, some experiments have been made; the obtained results are illustrated in Ref. [48]. In these experiments, various lengths for the recording time have been tested when the prototype was running at three different test speeds (30 mph, 50 mph, and 80 mph, corresponding to 48.28 km/h, 80.46 km/h, and 128.75 km/h, respectively). Such results prove that, in general, the SNR increases when the recording time increases; however, this leads to progressively losing spatial localization and to generating a non-stationary transfer function. Moreover, the SNR decreases as test speeds get larger. The duration of the time window has been selected in a way to achieve an accuracy equal to 8 in. (corresponding to 20.3 cm) in spatial localization at the various test speeds. The values evaluated for this duration are listed in Table 3.  
Additional work is in progress to quantify and improve the reliability of the defect detection prototype by (a) determining the optimal value of the threshold for the DI; (b) minimizing the effects of poor signal reconstruction; (c) improving the strength of the source excitation signal at low speeds [66].