## 6 Experimental Results
### Performances of Static ML Models  
Three well-known tree-based machine learning models are utilized to benchmark the performances of static ML models on the training and testing data. Specifically, we choose the Decision Tree, Random Forest and LightGBM models. Each model is configured with a maximum depth of 10 and a minimum samples split of 10, with a random state set to 42. The performances of these different static machine learning models were explored to identify the best-performing algorithm based on accuracy for both the training and testing datasets. These models are static, meaning that they do not adapt to changes in data distributions. To assess whether a model's performance deteriorates when subjected with new test data that have a different distribution from the training data, we calculated the cumulative accuracy over time. This approach helps in identifying how the accuracy of predictions evolves as the model encounters more data points. The cumulative accuracy at any given point is determined using the ratio of the number of correct predictions up to that point to the total number of predictions made. Mathematically, cumulative accuracy over time for the combined (training + testing) dataset is defined as  
\[\text{Cumulative Accuracy}=\frac{\sum_{i=1}^{n}I(y_{i}=\hat{y}_{i})}{n}\]  
where the variables are defined as follows:  
* \(y_{k}\) is the true label for the \(k\)-th data point. * \(\hat{y}_{k}\) is the predicted label for the \(k\)-th data point. * \(\text{I}(y_{i}=\hat{y}_{i})\) is the correctness indicator that outputs \"True\" if the prediction is correct (i.e., the predicted label matches the true label), and \"False\" otherwise. * \(i\) is the current data point index, ranging from 1 to the total number of samples in both the training and testing datasets combined. Figure 13 shows the performances of the static ML classification models on Fumehood dataset. As observed in Figure 13, the continuous learning model of TWIN-ADAPT consistently outperforms Random Forest and Decision Tree in both the training and testing phases, identifying it as the best static model. A notable drop in accuracy is observed for all classifiers when transitioning from the training to the testing data. This decline occurs because the samples in the training set have a similar statistical distribution, but the test set introduces a different statistical distribution. Consequently, the models trained on the training data cannot accurately detect anomalies or attacks in the test data. This phenomenon, known as concept drift, results in a significant decrease in accuracy when the test set begins. The performance is expected to degrade further if newer, more varied distributions are introduced into the testing data, demonstrating the limitations of static models in dynamic environment. Figures 14 and 15 show the performances of the static models for anomaly classification on the Vacuum Pump and Lithography Unit datasets. In both cases, the optimized and continuous learning-driven LightGBM model of TWIN-ADAPT maintains an accuracy comparable to those of Random Forest and Decision Tree during the training phase. However, a significant decline in accuracy is observed for all classifiers at the start of the \begin{table}
\begin{tabular}{l l l} \hline \hline
**CPS Dataset** & **Feature** & \(p\)**-Value** \\ \hline \multirow{3}{*}{Fumehood} & Temperature & 0.00066 \\  & Humidity & 0.016 \\  & Airflow & 0.001 \\ \hline \hline \multirow{2}{*}{Vacuum Pump} & Temperature & 0.0028 \\ \hline \multirow{2}{*}{Lithography Unit} & Temperature & \(8.789\times 10^{-26}\) \\  & Humidity & 0.079 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: \(p\)-values for KS-Test to measure difference in anomaly distributions between training and testing data for each feature of CPS dataset. testing phase, marked by the black dot. This drop indicates concept drift, as the test data introduce a different statistical distribution in anomalies, causing all models to struggle with maintaining their accuracy levels.