## 3 Datasets for the Attack and Anomaly Detection
### TON_IoT Dataset Analysis  
The TON_IoT dataset is created by the Intelligent Security Group of the UNSW Canberra, Australia, and positioned by its authors as realistic telemetry datasets of IoT and IIoT sensors. It contains data from seven IoT devices, namely, a smart fridge, GPS tracker, smart sense motion light, remotely activated garage door, Modbus device, smart thermostat, and weather monitoring system. All the data were generated using a testbed of Industry 4.0/Industrial IoT networks developed by the authors. The data include several normal and cyber-attack events, namely, scanning, DoS, DDoS, ransomware, backdoor, data injection, cross-site scripting, password cracking attacks, and man-in-the-middle. The TON_IoT dataset incorporates the ground truth indicating normal and attack classes for binary classification, and the feature indicating the classes of attacks for multi-classification problems. Statistics on class balance for device samples from the TON_IoT dataset are presented in Table 1.  
[PERSON] et al. and [PERSON] [8] also tried several popular machine learning methods to show that the TON_IoT dataset may be used to train classifiers for intrusion detection purposes. To justify the results and ensure that attacks are indeed identifiable, we have tried to follow the course of the authors' experiment with binary classification. It should be mentioned that the authors reported very high accuracy for the majority of the investigated methods (more than 0.8 for the F-measure in most cases). As we tried to follow the authors, at first we applied the same preprocessing procedures, namely, transformed categorical features with two unique values into binary ones, applied the min-max scaling technique to numeric features, and randomly split data into train and test subsamples in 80% to 20% stratified proportion. It should be noted that during data preprocessing, we found several artefacts in the data. For example, '_temp_condition_' feature for the fridge contains values 'high', 'low', 'low', \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**IoT Device** & **Normal** & **Attack** & **Total** & **Class Balance, \%** \\ \hline Fridge & 35,000 & 24,944 & 59,944 & 58/42 \\ Garage Door & 35,000 & 24,587 & 59,587 & 59/41 \\ GPS Tracker & 35,000 & 23,960 & 58,960 & 59/41 \\ Modbus & 35,000 & 16,106 & 51,106 & 68/32 \\ Motion Light & 35,000 & 24,488 & 59,488 & 59/41 \\ Thermostat & 35,000 & 17,774 & 52,774 & 66/34 \\ Weather & 35,000 & 24,260 & 59,260 & 59/41 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The statistics on the TON_IoT dataset class balance by devices. 'high', 'low', 'high' values, and '_sphone_signal_' for fridge contains 'true', 'false', '0', '1' values. As there are no special notes about that in the paper or the dataset description, we supposed that those were inaccuracies in the data and fixed them. Figure 1 shows the correlation between features for different devices, both with each other and with the anomaly behaviour label. We can note a high correlation between the features of the dataset for a fridge, garage door, GPS tracker, and motion light. At the same time, the correlation value between these features and the label is low. The correlation of features for Modbus, thermostat, and weather is close to zero. We applied the same machine learning models to those mentioned in the original paper, namely, Logistic Regression (LR), Linear Discriminant Analysis (LDA), k-Nearest Neighbour (kNN), Classification and Regression Trees (CART), Random Forest (RF), Naive Bayes (NB), Support Vector Machine (SVM), and Long Short-Term Memory (LSTM), with the hyperparameters that authors specified, and also tried to tune those hyperparameters using 4-fold cross-validation. We did not manage to reach the reported accuracy for most of the datasets, in either case. Table 2 shows the best values for F-measure that we received for classifiers trained on 80% of the data for each device calculated on the remaining 20% of the data. The best F-measure values were obtained for the GPS Tracker dataset.