## 4 Methodology: Distributed ML-Aided cyberattacks Detection on IoT Nodes
### Bounding Complexity for Neural Network on IoT  
Generally, data flow and computations of ML routine are easily incorporated as a singular system on personal computers or large-scale servers. From the perspective of the IoT ecosystem, it is clear that not all of the operations of ML routine can be fully deployed and utilized on micro-controllers. First, two sub-routines of the _Training_ phase mentioned above are not computationally heavy. For example, parameters such as file size can be extracted using system functions and be stored in a separate features list as a numerical value. The same stands for the first two sub-routines of the _Testing_ phase since those are nearly identical in both phases (feature reduction/selection is optional) [27].  
However, from Figure 3, we can see that the most computationally heavy part is considered to be the third step, _Model Training_, when the hypothesis evaluation and parameters optimization happens. ANN training is performed in two steps. The _output calculation step_ and _weights update step_ will be as follows in case of sigmoid activation function [50]:  
\[y_{actual}=sigmoid[(x_{1}\cdot w_{1}+\dots+x_{N}\cdot w_{N}+b)-\theta] \tag{1}\]  
\[w_{new}^{i}=w_{old}^{i}+\alpha\cdot(y_{target}-y_{actual})\cdot x_{i} \tag{2}\]  
Figure 2: Machine Learning application for cybersecurity hardening in IoT ecosystem.  
**Computational complexity:** the first step will take \(N\) multiplications, \(N\) additions and 4 operations for activation function where \(N\) is a number of features. The second step will take approximately 4 computing operations.  
**Space complexity:**\(N\) weights (\(w_{1}\cdot w_{N}\)), a bias \(b\), a threshold value \(\theta\) and output function value \(y\). Moreover, there has to be stored \(D\) training data samples with \(N\) features each and \(D\) class labels. In overall, it approximately requires \(D\cdot(2\cdot N+4)\cdot M+D\cdot 4\cdot N\cdot M\) computing operations, where \(M\) is a number of training epochs.  
Once the model is trained from the input data, the internal structure of the ML model is fixed as a set of weights/parameters. During the _Testing_ phase, \(2\cdot N+4\) parameters estimation is used against previously defined features to determine a class of the input data sample. From the storage complexity perspective, it will require \(N+2\) memory units for the parameters of the ANN and \(N\) parameters of a single unclassified data sample.  
In the real world, we can see that training ANN might be a way to heavy task for the IoT ecosystem components. Considering the aforementioned justification, training of a single-layer ANN might have polynomial complexity of \(O(N^{3})\), while training will be nearly linear \(O(N)\). For example, using KDD Cup 1999 full data set, it requires nearly 1110 seconds to train a Neuro-Fuzzy model on a powerful server station, while the testing phase requires \(95\times 10^{-6}\) seconds per new data sample to predict the actual class as was demonstrated in [PERSON] et al. (2019). Therefore, it is important to see that the heaviest parts of the computations in ML routine, i.e., _Training_, should be placed off IoT nodes to IoT gateways, while the prediction, i.e., _Testing_, can still be left at the IoT node level.