# Visualization Assisted Approach to Anomaly and Attack Detection in Water Treatment Systems
## 5 Experiments and Discussion
### Discussion  
The analysis of the experimental results with the machine learning models that are given in Figure 6 showed that the performance metrics are very high for some classes of attacks. To avoid overfitting during the experiments, the initial data set was divided into training and testing subsets (with respect to 70/30). This was applied in all series of experiments, including data sets with attacks of the first and second classes. The possible reason for such a high detection rate is the low complexity of these attacks and the high separability of the classes. The latter is proved by good visibility of the periods with attacks when they are visualized using the proposed graphical model. With an increase in the attack complexity as well as their number in the input data set, the performance metrics of machine learning methods slightly decrease but still remain quite high. This proves the efficiency of the proposed solutions.  
The implemented series of experiments showed that the proposed visualization technique allows constructing the graphical patterns of the anomalous behavior of the water treatment system. These patterns are formed by the _Change_Measure_ metric and characterized either by the values of the change metric or by the change rate of these values. It allows identifying points when the functioning of the system changes, which could be used as a trigger to start manual and/or machine-assisted checking of the system state.  
Nevertheless, the conducted experiments revealed the limitations of the approach. It was shown that it is required to apply different pre-processing techniques such as median filtering and window size setting in order to reveal different attack scenarios. To solve the first problem, the authors currently recommend implementing the following steps:  
1. Visualize the \"raw\" plot of the _Change_Measure_ metric for the whole set of attributes.  
Figure 11: The visualization of the _Change_Measure_ metric constructed for the data set that contains all five different types of anomalies. The upper plot shows if the system is under attack or not, and if the system is under attack it shows the class of the attack.  
2. Apply filter \"sliding average filter\" to highlight time periods when the rate of the metric change is higher in comparison to others.
3. Apply filter \"sliding median value\" to highlight time periods when the values of the metric are higher in comparison to others
4. If no signs of anomaly are detectable, group attributes based on their semantic relationships and perform steps 2-3.  
The comparative analysis of the proposed approach with the existing ones showed that with regard to visualization, the proposed approach is close to the solutions described in [19; 21]. Similarly to [21], the authors use a timeline to monitor the system's behavior and machine learning behavior. However, to construct the line chart, the authors introduce a specific preprocessing of data from multiple sensors that, in some sense, is close to [19; 20]. Unlike [20], the authors represent the behavior of the whole system by a point in a multidimensional space and then apply the Delaunay triangulation to assess the amount of changes in system behavior at a given moment of time. As the proposed _Change_Measure_ metric incorporates a set of system parameters, the developed visualization is able to form situational awareness of the water treatment system state.