## 3 Methodology
### Classification
#### 3.5.8 Softmax Layer
A training and validation graph of the proposed CNN is shown in Figure 4, where the consistent confidence of validation accuracy is shown. However, the accuracy graph is saturated over a few epochs. Various approaches using hyperparameters optimization, such as changing the learning rate (LR), the activation function, and epochs, are considered. Finely, the fine-tuned parameters for case study are shown in Table 4.  
Figure 4: CNN training graph. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Number** & **Layer Names** & **Activations** & **Kernel Size** & **Stride** & **Parameters** & **Feature Maps** \\ \hline \multirow{3}{*}{25} & Maximum & \multirow{3}{*}{\(16\times 16\times 512\)} & \multirow{3}{*}{\(2\times 2\)} & \multirow{3}{*}{\(2\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} \\  & pooling layer & & & & & \\  & (MP6) & & & & & \\ \hline \multirow{3}{*}{26} & Convolutional & \multirow{3}{*}{\(16\times 16\times 1024\)} & \multirow{3}{*}{\(3\times 3\)} & \multirow{3}{*}{\(1\)} & Weights = \(3\times 3\times 512\times\) & \\  & layer (C7) & & & & & \\  & & & & & & \\ \hline \multirow{3}{*}{27} & Batch & \multirow{3}{*}{\(16\times 16\times 1024\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & Offset = \(1\times 1\times 1024\) & \\  & Normalization (BN7) & & & & & Scale = \(1\times 1\times 1024\) & \\ \hline \multirow{3}{*}{28} & ReLU (R7) & \(16\times 16\times 1024\) & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} \\  & & & & & & \\ \hline \multirow{3}{*}{29} & Maximum & \multirow{3}{*}{\(8\times 8\times 1024\)} & \multirow{3}{*}{\(2\times 2\)} & \multirow{3}{*}{\(2\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} \\  & pooling layer (MP7) & & & & & \\ \hline \multirow{3}{*}{30} & Fully connected & \multirow{3}{*}{\(1\times 1\times 15\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & Weights = \(15\times 65\),536 & \\  & layer FC1 & & & & Bias = \(15\times 1\) & \\ \hline \multirow{3}{*}{31} & Softmax & \multirow{3}{*}{\(1\times 1\times 15\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} \\ \hline \multirow{3}{*}{32} & Classification & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} & \multirow{3}{*}{\(/\)} \\  & Layer & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: _Cont._As discussed above, the training parameters of the proposed CNN are shown in Table 4. It is observed that max-epochs are taken as 500, which are manually stopped due to the saturation of change in validation accuracy. The initial learning rate is set to 0.0004, where the frequency of iteration is taken as 65. Considering 13,000 iterations in total, by looking into a consistent accuracy rate from 1000 to more than 6000, training was manually stopped after 6764 iterations. At this time, epochs reached 105. After obtaining poor results of classification on validation and testing data, the state-of-the-art deep features using self-activations of CNN are applied. The self-activations are referred to as no other pre-trained model is used to extract deep features where the proposed CNN based transfer learning is performed on its fully-connected layer.