## 7 Experiments
### Learning Rate Adjustment Strategy and Loss Fuction  
The learning rate of the NN in the initial training stage was set at \(1\times 10^{-4}\). When the network was unable to continue converging, the learning rate was lowered. The learning rate lowering strategy varied according to the model and the two frequently used modes in this study are described as follows. One was the simplest and most commonly used strategy, which reduces the learning rate to 0.1 times the current rate when the loss cannot steadily decrease for more than 20 epochs at the current learning rate. In other words, starting from \(1\times 10^{-4}\), the learning rate is lowered to \(1\times 10^{-5}\), \(1\times 10^{-6}\),......, \(1\times 10^{-8}\), where the process ends because the rate cannot be lowered further. The other mode is to slowly lower the learning rate; when the loss cannot decrease at a learning rate of \(1\times 10^{-4}\), for every N_epoch epochs, the learning rate is reduced \(\alpha\) times. Specifically, N_epoch is an integer between 3 and 8, whereas \(\alpha\) is a real number between 0.92 and 0.8. These two values vary depending on the model.  
For understanding the quality of the model training, this paper applied the loss function to evaluate the prediction error of the model; if the predicted error is too large from the actual result, the loss function will be large. In the model training stage, the loss function gradually reduces the prediction error while the model is learning.  
This paper used the cross entropy as the loss function. Compared with other loss functions, the gradient of cross entropy decreases faster.  
\begin{table}
\begin{tabular}{c c c} \hline  & **Deep Neural Network** & **Batch Size** & **Code** \\ \hline
1 & Wide ResNet(N = 2, k = 4) & 128 & WRN\_k4 \\ \hline
2 & Wide ResNet(N = 2, k = 8) & 128 & WRN\_k8 \\ \hline
3 & Wide ResNet(N = 2, k = 4) & 256 & WRN\_b256 \\ \hline
4 & Wide ResNet(N = 2, k = 4) & 128 & WRN\_kernel5 \\  & Some kernels were enlarged to 5 \(\times\) 5. & 128 & WRN\_ex3D \\ \hline
5 & Wide ResNet(N = 2, k = 4) & 128 & WRN\_ex3D \\  & the front. (channel = 16, mean) & 128 & WRN\_ex3D\_v3 \\ \hline  & Wide ResNet(N = 2, k = 4) & 128 & WRN\_ex3D\_v2 \\
6 & The first 2D convolution layer was removed. & 128 & WRN\_ex3D\_v2 \\  & A 3D convolutional structure was added to the front. (channel = 24, mean) & 128 & WRN\_ex3D\_v2 \\ \hline  & Wide ResNet(N = 2, k = 4) & 128 & WRN\_ex3D\_v3 \\
7 & The first 2D convolution layer was removed. & 128 & WRN\_ex3D\_v3 \\  & A 3D convolutional structure was added to the front. (channel = 24, learnable) & \\ \hline \end{tabular}
\end{table}
Table 6: DNN labels.  
\begin{table}
\begin{tabular}{c c} \hline  & **Experiment Platform** \\ \hline CPU & Intel i7-7700 \\ RAM & 32 G \\ GPU & NVIDIA GTX 1080-8G \\ OS & Ubuntu 16.04 (in NVIDIA-Docker) \\ \hline \end{tabular}
\end{table}
Table 5: Experiment platform specifications.  
In the classification application, each data point has a set of predicted probabilities, so when computing the cross entropy of each data point, the entropy computed by each category is added, as shown in Equation (2):  
\[H=\sum_{c=1}^{C}\sum_{i=1}^{n}-y_{c,i}log_{2}(p_{c,i}) \tag{2}\]  
where \(H\) represents the cross entropy, \(n\) is the number of data points and \(c\) means the category, \(y_{c,i}\) is a binary indicator which is the \(i\)-th data point belongs to the real category of the \(c\)-class, \(p_{c,i}\) represents the probability of the \(i\)-th data point belongs to the \(c\)-class prediction.