# Performance Comparison of Machine Learning Disruption Predictors at JET
## 4 Discussion and Conclusions  
An indicator of the performance of more immediate readings in the prediction of disruptions is the accumulated fraction of detected disruption as a function of the warning time \(\Delta t_{warning}\). It provides the value, per unit, of successful alarms activated by at least the corresponding \(\Delta t_{warning}\) in advance, giving also, in a unique graph, a general overview of the premature detections and the alarm anticipation times. Moreover, it allows the reading of the successful prediction fraction (SP), which corresponds to the intersection between the accumulative curve and the minimum anticipation time (\(t_{D}-t_{value}=10\) ms at JET, red dashed vertical line), and the TD fraction as 1-SP. This is also a powerful means for comparing different models. Figure 9 reports such a comparison for the three proposed ML DP models. It is possible to see that the GTM (red line) has the earliest warning times, and its cumulative distribution of alarms is often to the right of \(t_{pre-disr-AUT}\), which is our target. These early alarms can be associated to the high number of false alarms of the GTM, which have a less smooth disruptive likelihood and need an assertion time to trigger the alarm. Then, the CNN and MLP-NN have similar cumulative distributions, with the MLP-NN which triggers one alarm more just before the red dashed vertical line, and the CNN which triggers some tardy alarms after it. The CNN follows the \(t_{pre-disr-AUT}\) distribution until 300 before the disruption, while the MLP-NN tends to stay to the left of the \(t_{pre-disr-AUT}\) \begin{table}
\begin{tabular}{|c|c|c|c|} \hline \hline  & & \multicolumn{2}{c|}{**Predicted**} \\ \cline{3-4}  & P + N = 257 & P = 110 & N = 147 \\ \hline \multirow{4}{*}{\(t_{pre-disr-AUT}\)} & \multirow{2}{*}{P = 108} & \multirow{2}{*}{TP = 102} & \multirow{2}{*}{FN = 6} \\ \cline{3-4}  & & & \\ \cline{2-4}  & N = 149 & & \\ \hline \multirow{4}{*}{Train} & PRECISION = 0.927 & \multirow{2}{*}{RECALL = 0.944} & \multirow{2}{*}{SPECIFICITY = 0.946} \\ \cline{3-4}  & ACCURACY = 0.946 & & \\ \cline{1-1} \cline{3-4}  & SP\% = 100\% & & \\ \cline{1-1} \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \cline{1-1} \cline{3-4}  & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Confusion matrix and performance indices of the MLP-NN prediction model evaluated on the test set. Figure 9: Accumulated fraction of detected disruptions by the MLP-NN (blue line), the GTM (red line), and the CNN (green line) models versus the warning time in the test set. The vertical red dashed line allows us to identify tardy detections. The dashed black curve refers to the \(t_{pre-disr-AUT}\). curve. The discharges where the three models missed the alarms are different from each other, but all except one pulse are characterized by a late mode-locking which causes the disruption. In the case of the GTM and of the FC-NN, the mode-lock absolute value is too low, and the predictors do not trigger an alarm, while in the CNN case, the mode-lock signal rise is not steep enough to trigger the alarm. Note that the statistical algorithm developed to automatically detect \(t_{pre-disr-AUT}\)[9] is non-causal and it cannot be used as a disruption predictor. Despite the better results in terms of performances, the MLP-NN and the CNN are mostly employed as black box algorithms and do not allow the extraction of significant information on the disruption type and possible recovery strategies, while the GTM allows the tracking of the position of the discharge and the instability mechanism to be associated with the position of the point in the map. Among the MLP-NN and the CNN, the latter provides an overall higher number of alarms and a generally higher warning time, keeping a low number of false alarms. Figure 10 reports, for the three predictors, the ROC curve.