## 4 Results
### Architecture of Modified AlexNet-33 (MAlexNet-33)
#### 4.2.3 The Combination of Layer and Parameter Modification  
Provided that uniformly scaling the fully connected layer parameters has proven beneficial, in this section, we combine this technique with the advantages of modifying the number of layers. This is done in two ways, the results for which are presented in Table 8: \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Activation Function** & **Activation Function** & **Accuracy** & **Network Size** \\
**1** & **2** & & \\ \hline ReLU & ReLU & 83.27\% & 157.92 MB \\ Leaky ReLU (0.01) & Leaky ReLU (0.01) & 84.32\% & 157.92 MB \\ ReLU & Leaky ReLU (0.01) & 85.38\% & 157.92 MB \\ Tanh & Leaky ReLU (0.01) & 73.49\% & 157.92 MB \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Successive Activation Function Combination Summary. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Activation Function** & **Input to** & **FC6** & **FC7** & **Num. of Layers** & **Scale** & **Epochs** & **Network Size** & **Weighted F1** \\ \hline ReLU & 9216 & 4096 & 4096 & 25 (orig \({}^{\star}\)) & None & 10 & 221.4 MB & 86.24\% \\ ReLU & 4608 & 574 & 574 & 25 (equ \({}^{\star}\)) & Equ. & 30 & 31.90 MB & 85.76\% \\ ReLU & 4608 & 576 & 256 & 25 (div 16) & 16 & 30 & 31.23 MB & 85.15\% \\ Leaky ReLU (0.01) & 4608 & 576 & 256 & 25 (div 16) & 16 & 30 & 31.23 MB & 85.48\% \\ Leaky ReLU (0.01) & 4608 & 384 & 172 & 25 (div 24) & 24 & 30 & 23.82 MB & 86.82\% \\ ReLU & 4608 & 384 & 192 & 25 [64] & None & 30 & 23.85 MB & 85.76\% \\ \hline \hline \end{tabular} \({}^{\star}\)orig—refers to the original AlexNet layer; equ—refers to using equal fully connected layer parameters.  
\end{table} 
Table 7: Parameter Modification Results. Decreasing the number of layers: Similar to the experiment conducted in Section 3.2.1, this reduces the number of convolutional and fully connected layers within the network. For example, MAlexNet-23 refers to the removal of conv4 and relu4, maintaining all fully connected layers. On the other hand, MAlexNet-20 is the same network structure examined in Section 3.2.1. * Increasing the number of layers: For this experiment, another grouped convolutional layer/s with the relevant activation function was added to the network structure. From the original AlexNet model, the grouped convolutions carry bias learnable weights of \(1\times 1\times 192\times 2\) and \(1\times 1\times 128\times 2\), respectively. For this work, additional grouped convolution functions were added, such that it has a bias learnable weight of \(1\times 1\times 64\times 2\) for MAlexNet-27, and \(1\times 1\times 64\times 2\) and \(1\times 1\times 32\times 2\) for MAlexNet-33. Accordingly, Leaky ReLu (0.01) activation functions were utilized for all grouped convolutional layers. As per Table 8, it is observed that the top performing algorithm is the MAlexNet-33, which is designed as a combination of both fully connected parameter scaling, as well as the addition of two new grouped convolutional layers with bias learnable weights of \(1\times 1\times 64\times 2\) and \(1\times 1\times 32\times 2\), and relevant activation layers. This provided a weighted F1-score of 87.96%, exceeding the performance of the AlexNet, with a network size of 14.33 MB. This suggests an over 95% decrease in the size of the resource requirements when compared to the original model. When compared to [66], this also improved both the performance and the network size, exceeding the performance by around 2.16%