# A Dense Neural Network Approach for Detecting Clone ID Attacks on the RPL Protocol of the IoT
## 4 Proposed Framework
### Unsupervised Pre-Training  
The topologies contained in each data set \(D\) conformed to IoT network packets, which can vary from different origins, devices, latency, and changing behaviours, that depend directly on the environment that is being observed. Although data exploration, transformation, and scaling are mandatory steps to construct proper inputs for the learning algorithm, feature engineering is a key step to select relevant information, reduce the computational costs from high-dimensional samples, and filter noisy data; inherent factors from IoT-RPL communications [53]. To reach adequate feature extraction and selection, in this work, we propose the use of an unsupervised learning algorithm, known as Autoencoder [54; 55], a data representation method capable of tracking network packet variations by backpropagation and reconstructing the output values as being equal to the initial inputs using an identity function; in other words, with minimum reconstruction error. A simple Autoencoder is depicted in Figure 8.  
The structure of an Autoencoder is composed of an input layer _Layer_\(L_{1}\), a hidden layer _Layer_\(L_{2}\), and an output layer _Layer_\(L_{3}\). For the most part, the hidden units in _Layer_\(L_{2}\) learn from a lower-dimensional representation of the input, as this layer has fewer units [56]. In the encoding phase, the hidden representation is compacted to a vector \(h_{d}\), mapped with an input vector \(x_{d}\) containing \(d\) features:  
\[h_{d}=f_{\theta(x_{d})=\sigma(Wx_{d}+b)}, \tag{2}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Data Set Name** & **No. of Features** & **Samples** \\ \hline _cloneid\_20n_ & 67 & 1,749,976 \\ _cloneid\_50n_ & 121 & 2,131,328 \\ _cloneid\_100n_ & 211 & 2,078,832 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Data set descriptions after the pre-processing steps.  
Figure 8: Conceptual diagram of an Autoencoder.  
where \(f_{\theta}\) is the mapping function, \(W\) is the \(p\times p\) weight matrix for the \(p\) hidden units, \(b\) is the bias vector, \(\theta\) is the mapping parameter set \(\theta=W,b\), and \(\sigma\) is the sigmoid activation function.  
At the same time, in the decoding phase, the reconstructed \(d-\)dimensional vector \(y_{n}\) is computed by mapping back the condensed hidden representation \(g_{\theta}\) of vector \(h_{n}\), as follows:  
\[y_{i}=g_{\theta(h_{n})=\sigma(Wh_{n}+b)}. \tag{3}\]  
Finally, to minimize the reconstruction error between input and output for \(m\) samples, the following model is computed:  
\[E(x,y)=\frac{1}{m}\Big{\|}\sum_{i=1}^{m}(x_{n}-y_{n})\Big{\|}^{2}. \tag{4}\]  
To find the latent representation of each data set \(D\), a vector \(h_{d}\) was used as a feature selector, through an \(\ell_{1}\) regularization task implementing a sparsity constraint (also called Sparse Auto Encoding; SAE), for the hidden layer. This regularization shrinks noisy feature weights towards zero and extracts the most relevant ones. The Autoencoder and SAE configurations, activation functions, epochs, and batch sizes are depicted in Figure 9.  
The configuration employed two-thirds of the total number of input neurons over the hidden layer; the Rectified Linear Unit (ReLU) activation function was used to simplify training and improve performance [57]; Adam was utilized as an optimizer, with a gradient descent algorithm to adapt the learning rate [49]; and \(\ell_{1}\) regularization with a coefficient of 0.0001 was established [58].