# Deep-Learning-Based Approach for IoT Attack and Malware Detection
## 3 The Proposed CNN  
In this paper, a novel 1D convolutional neural network (CNN) architecture tailored for efficient classification tasks is presented. The architecture is proposed as a foundation for a lightweight model ensuring its suitability for real-time and resource-constrained applications. Additionally, the architecture is scalable to handle large-scale datasets, maintaining its effectiveness and robustness. The proposed 1D CNN comprises 75 layers (see Figure 2), organized into several key blocks: the input block, convolutional blocks, self-attention blocks, and the output block. The improvements in this model are achieved through the following structural components:  
* _Convolutional Layers_: These layers learn spatial relationships in the data and extract various features. By stacking multiple convolutional layers, the model can capture complex patterns in the IoT data;
* _GELU Activation Function_: The Gaussian Error Linear Unit (GELU) activation function is used to enhance the model's ability to learn non-linear relationships. This activation function is defined as \[GELU(x)=x\cdot\phi(x)\] (2) where \(\phi(x)\) is the cumulative distribution function of the standard normal distribution; \begin{table}
\begin{tabular}{c l c c c} \hline \hline
**No** & **Class Name** & **Train** & **Test** & **Validation** \\ \hline
1 & BENIGN & 1,591,168 & 340,965 & 340,964 \\
2 & DDoS & 89,619 & 19,204 & 19,204 \\
3 & PortScan & 111,251 & 23,840 & 23,839 \\
4 & Bot & 1376 & 295 & 295 \\
5 & Infiltration & 25 & 5 & 6 \\
6 & Web Attack Brute Force & 1055 & 226 & 226 \\
7 & Web Attack Sql Injection & 15 & 3 & 3 \\
8 & Web Attack XSS & 456 & 98 & 98 \\
9 & FTP-Pataor & 5557 & 1191 & 1190 \\
10 & SSH-Patator & 4128 & 885 & 884 \\
11 & DoS GoldenEye & 7205 & 1544 & 1544 \\
12 & DoS Hulk & 161,751 & 34,661 & 34,661 \\
13 & DoS Slowhttptest & 3849 & 825 & 825 \\
14 & DoS slowloris & 4057 & 869 & 870 \\
15 & Heartbleed & 8 & 2 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The number of training, test, and validation data for different attack types in the CIC-IDS2017 dataset. * _Self-Attention Mechanism:_ Inspired by the superior performance of self-attention mechanisms, multiple self-attention layers are incorporated into the architecture. The self-attention mechanism helps the model focus on important features of the data, improving overall accuracy. The self-attention is computed as \[Attention(Q,K,V)=softmax\bigg{(}\frac{\theta K^{T}}{\sqrt{d_{k}}}\bigg{)}V\] (3) where Q is the query matrix, K is the key matrix, V is the value matrix, and d\({}_{k}\) is the dimension of the key;
* _Layer Normalization and Batch Normalization:_ These normalization techniques stabilize the learning process and enhance the overall performance of the model. Layer normalization is applied after each block, ensuring that the inputs to each layer have a mean of zero and a variance of one;
* _Dropout Layers:_ Dropout is used to prevent overfitting and enhance the model's generalization capability. During training, dropout randomly deactivates a portion of input units, which helps in regularizing the model;
* _Global Max Pooling:_ This layer reduces the spatial dimensions of the input, retaining the most important features and reducing computational load. The underlying theory behind the proposed model is based on the effectiveness of convolutional neural networks in capturing spatial hierarchies in data, combined with the power of self-attention mechanisms to focus on relevant features. The model's structure is designed to achieve a balance between computational efficiency and high classification performance, making it suitable for a wide range of IoT applications. The proposed model begins by taking a sequence input, which is then processed through a 1D convolutional layer with 96 filters, a kernel size of 3, and a stride of 1, maintaining the same padding to preserve the input dimensions. Following this, layer normalization is applied to stabilize and accelerate the training process.