# A Deep Learning Method for Lightweight and Cross-Device IoT Botnet Detection +
## 5 Results
### Comparison with the Related Proposals Using N-BaloT  
It is worth noting that N-BaloT is a widely used dataset in the IoT-based anomaly detection field. Consequently, we can compare the metrics obtained in our study with other similar proposals in the literature. Authors in [2; 3; 24], as mentioned in Section 2, assess different state-of-the-art anomaly detection methods on the same N-BaloT dataset used in our paper. However, different from our study, which shows the performance metrics per IoT device, the authors report the performance figures per attack. For example, in [3], the authors obtain a recall of 0.99 for the gafgyt combo attack and of 1 for the mirai_ack attack. Again, the authors of the detector proposed in [26] show only the ROC curves--and AUC values--as performance figures; therefore, they cannot be directly compared with our results. As for the use of the autoencoder, this type of neural network is leveraged to perform automatic feature extraction with the aim of reducing the dimensions of the data being processed. Therefore, differently from our study, the autoencoder is simply a building block of a more complex detection architecture. As for the federated-based approach proposed in [27], the recall values are below 0.75 under the 50 epoch range; the score jumps up to 0.99 from 60 epochs onward and remains at that high level while using the federated learning approach, for all experiments. These results are in line with our recall values. The performance obtained in DeL-IoT [28], instead, is comparable to the ones obtained in our study. However, the authors use the autoencoder to perform feature extraction and not anomaly detection (as intended in our study). As for the approach described in [30] mentioned in Section 2, tested using the N-BaloT dataset, the best recall--0.999--is found for the Samsung SNH 1011 N webcam device. We also obtained a recall equal to 0.9994 for this device. Similar values of recall are preserved for all the IoT devices in our study, different from the results shown in [30]. Moreover, according to the figures reported in [30], the best FPR--0.009--is obtained for the Ecobee thermostat device. For the other devices, the FPR ranges from 0.016 to 0.098. The FPR obtained with our approach, instead, ranges from 0.0 to 0.0071.  
The results obtained with our cross-device method are inline--if not even better--with those obtained by other anomaly detection techniques assessed through the N-BaloT dataset. Besides the inherent benefits of deep learning, the improvement over existing methods is explained by the interplay of **two factors**, which are the novel contributions of this work. The former is the cross-device notion: learning a single model atop the normal traffic related to different IoT devices allows for the recognition of a greater number of nominal behavioral patterns and related variants. This makes our approach valuable in terms of adaptability and transferability across the devices, and it leads to better performance. The latter is the proposed outlier-based threshold selection method. In this respect, there exist several \"fixed\" criteria, such as _mean_, _median_, _mean plus standard deviation_ and _nth percentile_, that can be found in the related literature to determine a threshold from normal data. For example, the authors in [2] consider the mean and the standard deviation to select the threshold. It should be noted that a \"fixed\" criteria approach may not be the best fit for all the datasets in hand. In order to overcome this limitation, we proposed a data-driven method that can adapt to potential outliers. Our method improves the results over traditional techniques because it allows mitigating the inherent imperfections of the training data and occasional inconsistencies of the autoencoders at providing good reconstructions also for outliers, which are well-known in the literature [37].