# Visualization Assisted Approach to Anomaly and Attack Detection in Water Treatment Systems
## 5 Experiments and Discussion
### Analysis of the Proposed Visualization Efficiency  
The goal of the experiments conducted with a visualization component was to determine the most suitable parameters for the calculation of the integral metric _Change_Measure_, i.e., size of sampling window that defines a number of the points used to construct Delaunay triangulation and enables clear identification of the anomaly. The authors also evaluated the impact of smoothing filters on the visual efficiency to reveal specific classes of attacks. The experimental data consisted of five data sets that are described in Section 5.1. To determine the optimal number of data points that are used to construct Delaunay triangulation, the authors calculated and plotted the _Change_Measure_ metric with the following parameters:  
* Distance between 2 points (parameter \(n=2\)),
* Triangular area by 3 points (parameter \(n=3\)),
* Delaunay triangulation by 5 points (parameter \(n=5\)),
* Delaunay triangulation by 10 points (parameter \(n=10\)). For the test data, the data set with the first class of the attack was used. Figure 7 shows plots of the _Change_Measure_ metric for these four settings, and the background color shows the time intervals with the attack. Figure 6: Machine learning method results comparison (f1-score). Figure 7: _Cont._  
It is obvious that the plots of the metric based on the calculation of triangular squares produce similar patterns of anomalous activity, while the metric based on the calculation of the Euclidean distance between two points gives a slightly different plot (Figure 7a). It is possible to see the transition period at the beginning more clearly, and the normal period of functioning is also characterized by periodical bursts in the metric values. In contrast, the plots of the metrics based on the triangles' area allow an analyst to detect normal and anomalous periods clearly: the periods of attack are characterized by high scatter in values and higher frequency of change, and the metric values for normal periods are almost close to zero and change slightly. As all plots produce similar results, the metric based on the calculation of the triangle square is preferable because it is faster to calculate and does not require the accumulation of data, which may be critical for the online monitoring of the system. In the next series of experiments, the authors used the _Change_Measure_ calculated with \(n\) set to 3. The next series of experiments was devoted to the evaluation of the efficiency of the _Change_Measure_ metric to determine different classes of the attack. An attack of the first class is clearly seen on the plot of the _Change_Measure_ metric with different visualization settings. Figure 8 shows these plots, it should be noted that the _Change_Measure_ metric is constructed for the whole set of attributes. The plot of the \"raw\" metric is characterized by a constant change of the metric values, the sliding filters smooth these changes making the start and stop points of the attack more visible. Figure 8: _Cont_. Figure 7: Plot of the _Change_Measure_ metric constructed for different settings of the \(n\) parameter: \(n=2\) (a), \(n=3\) (b), \(n=5\) (c), and \(n=10\) (d). The attack of the second class was not easy to detect as there were no clearly visible changes in the behaviour of the _Change_Measure_ metric when it was constructed for the whole set of attributes. Thus, it was decided to focus only on the parameters that could be potentially impacted by the attack, i.e., parameters that characterize the volume of the water in the reservoir. This allowed authors to reveal some anomalous spikes in the system's functioning. Figure 9 shows the plot of the obtained values of the _Change_Measure_ metric. It is clearly seen that there is a sequence of spikes at the beginning of the plot that could indicate that the system is in a transient state, reaching its normal functioning mode, and there are four single outliers.