# A Deep Learning Method for Lightweight and Cross-Device IoT Botnet Detection +
## 4 Dataset, AE Design and Implementation
### Reference Dataset and Partitioning  
The dataset considered in this paper is **N-BaIoT**[2]. It provides a public botnet IoT dataset available at the **University of California at Irvine (UCI)** machine learning repository. The authors deployed all of the components of two botnets in an isolated lab and used them to infect nine commercial IoT devices listed in Table 1. All the devices execute the attacks of two botnets--namely Mirai and BASHLITE--that have previously infected the IoT devices. For each device, the data were obtained under both normal operations and attack conditions. The dataset consists of fixed-length records of features computed from the \begin{table}
\begin{tabular}{l l} \hline \hline
**IoT Device** & **Category** \\ \hline Dammini & Doorbell \\ Ecobee & Thermostat \\ Ennio & Doorbell \\ Philips B120N/10 & Baby Monitor \\ Provision PT-737E & Security Camera \\ Provision PT-838 & Security Camera \\ Samsung SNH 1011 N & Webcam \\ SimpleHome XCS7-1002-WHT & Security Camera \\ SimpleHome XCS7-1003-WHT & Security Camera \\ \hline \hline \end{tabular}
\end{table} 
Table 1: IoT devices in the N-BaIoT dataset. Figure 2: Steps underlying the threshold selection method. lower-level network activity. In particular, each record is identified by 115 features with a systematic structure plus a class label (e.g., \"normal\" or \"TCP attack\"). The features model traffic statistics over several temporal windows. Each statistic is further temporally aggregated using a weighted sum that progressively leads to the decay of the oldest contributions to the sum. It is worth pointing out that botnet infections consist of multiple steps, such as propagation, bot infection, communication with C and C server, and performing other types of malicious activities [36]. According to [2], it is not sufficient to determine only the early stages of infection. The data contained in the N-BaIoT dataset pertain only to the last stage of botnet construction, when IoT bots begin launching attacks. Interested readers are referred to [2] for any additional information on the N-BaIoT dataset. Our experiments are based on all the devices described in the N-BaIoT dataset. Experiments are performed in a binary classification scenario. All the records produced within different types of attacks are considered as belonging to a unique generic class named ATTACK--encoded with the numeric label 1--whereas NORMAL records are assigned 0 as label. It is worth remarking that the N-BaIoT dataset is organized into _separate datasets_, each containing both normal and attack traffic corresponding to a single IoT device. For our experiments, we split the original datasets into three **disjoint splits**: _training set_, _validation set_ and _test set_. While splitting the dataset corresponding to an IoT device, we preserve the original sequence of the records because--as said above--the features of N-BaIoT are based on the temporal windows and are aggregated using weighted sums. Moreover, each record of the original dataset is assigned to a unique split. For each IoT device, we obtain:  
* _Training set_. It contains 70% of the total NORMAL records; moreover, 10% of the training set, according to the threshold selection criteria described in Section 3, is the \"threshold set\", meant for the threshold selection process;
* _Validation set_. It contains 15% of the total NORMAL records as for any machine learning experiments, it is used to provide an unbiased evaluation of the model in order to find the optimal values for the hyper-parameters;
* _Test set_. It contains 15% of the total NORMAL records and all ATTACK records. The records in this set are accompanied by the corresponding labels that are used to assess the correctness of the predictions. Table 2 shows the cardinality of the sets for the first group of experiments, i.e., separate autoencoding.