## 6 Results
### Classification Performance
#### 6.1.3 Using Different CNN Architectures  
To even more exhaustively evaluate the performance of CNN-TL-DE algorithm, we evaluated the performance against two additional commonly used CNN architectures: Inception-V3 [59] and DenseNet-121 [60].  
The Inception-V3 CNN architecture was first introduced by [PERSON] et al. in 2015 [61], where the authors presented GoogleNet architecture, which would serve as an alternative to VGGNet with  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Metric** & **CNN-TL-GA** & **CNN-TL-PSO** & **CNN-TL-DE** \\ \hline accuracy & 79.03 \(\pm\) 6.18 & 78.07 \(\pm\) 5.89 & **81.31 \(\pm\) 3.51** \\ AUNP & 85.98 \(\pm\) 4.17 & 85.41 \(\pm\) 3.89 & **87.56 \(\pm\) 2.30** \\ precision & 82.06 \(\pm\) 3.49 & 81.30 \(\pm\) 5.89 & **83.13 \(\pm\) 3.33** \\ recall & 78.74 \(\pm\) 6.49 & 78.11 \(\pm\) 5.78 & **81.30 \(\pm\) 3.41** \\ \(F\)-1 & 80.32 \(\pm\) 4.98 & 79.62 \(\pm\) 4.19 & **82.20 \(\pm\) 3.31** \\ kappa & 71.97 \(\pm\) 8.31 & 70.77 \(\pm\) 7.81 & **75.08 \(\pm\) 4.64** \\ time & **3311.46 \(\pm\) 271.46** & 3667.14 \(\pm\) 272.13 & 3430.43 \(\pm\) 315.78 \\ epochs & **744.7 \(\pm\) 66.09** & 859.5 \(\pm\) 79.45 & 790.3 \(\pm\) 85.49 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparison of classification performance results with alternative optimization techniques.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Friedman Test** & \multicolumn{2}{c}{**Wilcoxon Signed Rank Test**} \\ \hline
**Metric** & **All Three** & **CNN-TL-DE vs. CNN** & **CNN-TL-DE vs. CNN-TL** \\ \hline accuracy & \textless{}0.001 & 0.005 & 0.033 \\ AUNP & \textless{}0.001 & 0.005 & 0.022 \\ precision & \textless{}0.001 & 0.005 & 0.022 \\ recall & \textless{}0.001 & 0.005 & 0.022 \\ \(F\)-1 & \textless{}0.001 & 0.005 & 0.022 \\ kappa & \textless{}0.001 & 0.005 & 0.022 \\ time & \textless{}0.001 & 0.005 & 0.005 \\ epochs & \textless{}0.001 & 0.005 & 0.005 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Statistical comparison of the three compared methods (all results are statistically significant).  
the ability to perform well under strict constraints on memory and computational budget [59]. The Inception-V3 architecture presented in [59] is an upgrade to the original one, where the authors introduced some general principles and optimization ideas such as factorizing convolutions with large filter sizes and utilization of auxiliary classifiers that proved to be useful for scaling up convolution networks in efficient ways. For the purpose of evaluation of our CNN-TL-DE algorithm, we adapted the Inception-V3 architecture in a manner where all but the last inception module layers are frozen, while the inception module layers are enabled for fine-tuning.  
The DenseNet-121 architecture was presented in 2017 by [PERSON] et al. [60]. The architecture features a simple connectivity pattern which enables the maximum information flow between the layers. To achieve this, the authors connected all layers with matching feature-map sizes directly with each other. In addition, each layer also obtains inputs from all preceding layers and passes on its own feature-maps to all subsequent layers, which enables to preserve the feed-forward nature [60]. We adapted the DenseNet-121 architecture in a way that the layers constructing a fourth (last) dense block are enabled for fine-tuning, while other layers are frozen.  
The obtained classification performance results on three different CNN architectures are summarized in Table 7. The results obtained on Inception-V3 lag somewhat behind, while the results on the other two are very similar, within the range of 1%, with the exception of consumed training time. While the predictive performance results on DenseNet-121 are a bit higher, they exhibit much higher variance. On the other hand, both Inception-V3 and especially DenseNet-121 consumed much more time for training than the VGG19 architecture.