## 4 Applications, Results, and Discussion
### Comparison of Prediction Results of Improved Versions of ELM  
As represented in Section 4.2, the ELM was identified to be the best machine learning approach. This section compares the improved versions of the ELM algorithm, namely, the online sequential ELM, bidirectional ELM, and weighted ELM, with respect to the prediction quality and performance measures. The results are analyzed according to their algorithm's characteristics. Because the performance of ELM affects the values of its parameters (i.e., activation function and number of hidden neurons), to evaluate the performance, the validation process of such parameters is necessary [52].  
In response to this, the sensitivity analysis of the parameters was conducted for the improved version of ELM algorithms. Based on the parameter sensitivity analysis, the applied activation function and number of hidden neurons of each algorithm are as follows: OS-ELM: triangular basis (Tribas) function and 10; B-ELM: sine function, and 10; W-ELM: sigmoid function and 75. The cyber-attack prediction results are presented in Table 7 and Figure 4.  
Table 7 highlights the three improved versions of ELM and the standard ELM, wherein all variants, including the standard ELM, achieved a ranking score \(S\) higher than (or close to) 0.9. The results of OS-ELM show the best overall performance (S = 0.910). The OS-ELM has the top score with respect to the classification score \(S_{\text{CLF}}\). Moreover, the detection history depicted in Figure 4a shows that all attacks are immediately detected, with the exception of the first attack scenario, which is detected a few hours after its starting time. However, in comparison with the other improved algorithms, false alarms are more likely to occur before an attack occurs. This is because the OS-ELM was developed considering real-time system operation or maintenance tasks, wherein the training data are accumulated over time. Therefore, it can be shown that the time difference results in false prediction as most of the false detections are generated intensively before an attack occurs. The performance of B-ELM is close to that of the standard ELM and W-ELM with respect to the \(S\) metric and identifying TTD. However, it is more likely to generate false alarms. Unlike other variants of ELM, B-ELM increases the user convenience in terms of determining the appropriate number of hidden nodes while providing accurate prediction results without the need to conduct sensitivity analysis for the number of hidden neurons.  
The W-ELM has a similar ranking score \(S\) to that of OS-ELM and B-ELM. The \(S_{\text{CLF}}\) value of W-ELM is lower than that of the others, resulting in a lower \(TPR\) and higher \(FN\). All of these lead to a score of \(S\) equal to 0.906. Moreover, the W-ELM algorithm is shown to have the least timing error, as presented in Table 7. This implies that although almost all the starting attack points can be detected using the W-ELM, it is also prone to FPs. This may be due to the weight of data in the training process. The weight of abnormal cases is affected by the cyber-attack detection. For this reason, most of the false predictions for the W-ELM occur close to the attack conditions.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Performance** & **ELM** & **OS-ELM** & **B-ELM** & **W-ELM** \\
**Indices** & & & & \\ \hline S & 0.891 & 0.910 & 0.900 & 0.906 \\ \(S_{\text{CLF}}\) & 0.841 & 0.880 & 0.865 & 0.832 \\ \(S_{\text{TTD}}\) & 0.941 & 0.940 & 0.935 & 0.980 \\ F1 & 0.764 & 0.806 & 0.768 & 0.744 \\ TNR & 0.959 & 0.952 & 0.958 & 0.951 \\ TPR & 0.722 & 0.808 & 0.732 & 0.713 \\ PPV & 0.810 & 0.804 & 0.808 & 0.777 \\ Accuracy & 0.913 & 0.924 & 0.914 & 0.904 \\ TP & 294 & 329 & 298 & 290 \\ FP & 69 & 80 & 71 & 83 \\ TN & 1613 & 1602 & 1611 & 1599 \\ FN & 113 & 78 & 109 & 117 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Comparison of performance results among the improved versions of ELM.