# Detecting DDoS Attacks in IoT-Based Networks Using Matrix Profile
## 4 Findings and Discussion
### Evaluation and Discussion  
In this section, we evaluate the proposed method (matrix profile) to detect DDoS attacks in terms of precision, accuracy, and runtime of the algorithm. In addition, this section discusses the results and explains them. First, we repeat one of the experiments in the previous section using different values of window sizes as window size can affect the accuracy of the matrix profile, as shown in the results. In addition, we use time-series data and Z-normalised time-series data as inputs to the proposed algorithm to calculate the matrix profile and explain which one is recommended.  
Figure 15 demonstrates the matrix profile calculation for time-series data (memory utilisation of SC) with different values of window size (3, 10, 25, 50, 100, 500, and 1000). Note that we begin with 3 because 3 is the smallest number that we can use as a window size in matrix profile. As can be seen, the algorithm successfully identified the anomaly in the data that occurred because of a DDoS attack. It is worth mentioning that the anomaly is clearer when using a larger window size. However, it should be kept in mind that the length of the matrix profile is equal to the data minus the window size. Therefore, if the window size is large, for example, 1000, then the attack occurred at the time of detection plus window size.  
On the other hand, Figure 15 illustrates the matrix profile for Z-normalised time-series data (memory utilisation of SC). This figure shows that window size can affect the accuracy of the matrix profile algorithm. The outcome shows that when the window size is small, the accuracy is negatively affected in the Z-normalised time-series data. This is because the Z-normalised data are less sensitive to change. Therefore, when using a small window size, the algorithm may fail to detect the anomaly in the data. However, the runtime of the matrix profile algorithm when using Z-normalised data is shorter.  
Figure 16 depicts a matrix profile algorithm runtime comparison between using time-series data and Z-normalised time-series data. The figure shows that there is almost no difference between using time-series data and Z-normalised data with matrix profile in terms of calculation time when the size of the data is small (around 10 K). On the other hand, the figure highlights that runtime using Z-normalised time-series data is much shorter than the runtime using the matrix profile algorithm with time-series data. Therefore, we recommended using Z-normalised time-series data when the size of the data is large.  
Table 1 shows the number of true positives and false positives from the experiment. It also shows the precision of the algorithm. Note that Equation (5) is utilised to calculate the precision of the matrix profile algorithm when using time-series data and Z-normalised time-series data with different values of window sizes [43]. The table shows that the precision of the matrix profile algorithm with time-series data is much better than that of using Z-normalised time-series data because the matrix profile algorithm becomes less sensitive to anomaly data when using Z-normalised data.  
\[\text{Precision}=\frac{\text{True Positive}}{\text{True Positive + False Positive}} \tag{5}\]  
Figure 14: Memory Utilisation in Surveillance Camera with additional traffic.  
Figure 15: Matrix profile for memory utilisation using different window sizes.  
In addition, paper [44] used an unsupervised machine learning algorithm to detect DDoS attacks in the same dataset. The accuracy of their algorithm is around 87%. However, the precision of this algorithm is varied from one DDoS attack to another. For example, the precision of their algorithm is 28% for UDP DDoS attacks, whereas it is 100% for TCP DDoS attacks. Comparing this result with our finding, our algorithm is data agnostic because it analyses the symptoms of the DDoS attack and not the traffic itself. Thus, the precision is the same, disregarding the type of DDoS attack. The precision of our algorithm varies from 80% to 100%.