## 3 Results
### The Final Deep Network Classifiers Structure  
The batch size was 4 and also they were accumulated 2 epochs before one step in optimization (resulting in an effective batch size of 8). The well-known CrossEntropyLoss \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Name**} & **\# of Total Samples** & **\# of Class Label 0** & **\# of Class Label 1** & **\# of Class Label 2** \\  & **(Before Augmentation)** & **(Healthy, Undamaged)** & **(8 h after Bruising)** & **(16 h after Bruising)** \\ \hline Train & 147 & 49 & 49 & 49 \\ Test & 21 & 7 & 7 & 7 \\ Validation & 42 & 14 & 14 & 14 \\ Total & 210 & 70 & 70 & 70 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Number (#) of samples in the dataset and in train/test/validation disjoint sets. Figure 11: Building block of MobileNet containing Depthwise Convolution and Pointwise Convolution boxes. \begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{1}{c}{**Model Name**} & **Parameter Set Size (MB)** & **Train Time Per Epoch (seconds)** \\ \hline ResNet & 242 & 24 \\ ShuffleNet & 5 & 7 \\ DenseNet & 43 & 46 \\ MobileNet & 12 & 7 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: 3D-CNN model parameter sizes and training time per epoch. loss (error) function was used as the optimization criterion. In addition, the Adadelta optimizer with an exponential learning rate was applied that helped in achieving faster convergence and increased stochastic nature of the training process. It was started with a learning rate of 0.1 and was then reduced exponentially by a gamma factor of 0.95. In order to use all the spatial information of the hyperspectral images, we concatenated the spectrums of each sample and created a 3D image. As a result, the input to our models is an image of size \(174\times 160\times 120\) pixels. One important hyper-parameter for our classifier in the training phase is the batch size. We had some limitations in the GPU memory storage (Google Colab. Environment). Additionally, the 3D-CNN models consume more memory storage space than 2D-CNN models because of the larger trainable parameters in these 3D model kernels and pooling layers. In order to overcome this limitation, we used a batch size of 4 with a gradient accumulation of two back propagations. This provides an effective batch size of 8 in our training phase, as already mentioned. We used CrossEntropyLoss cost function to compute the error of the models. The last layer of the classifier uses a Softmax layer to provide a prediction probability for each of the classes. By using the cross-entropy loss, these logits (probabilities) are compared with the expected label vector which is a one-hot vector with one single non-zero value on the index of the expected label and zero on all other indices of the vector. As a result, if an error is introduced on these logits, it will go back to update model parameters through the backpropagation learning algorithm. As the training optimizer, we used Adadelta [31]. Adadelta adapts the learning rate dynamically over time and has two major benefits: there is no need to manually select a global learning rate and addresses the problem of the continual decay of learning rates throughout training. We used an exponential scheduler for our optimizer and it helped the mode to faster converge to an optimal point in the loss function. We started with a base value of 0.1 for the learning rate and after each training epoch, this learning rate is multiplied by a gamma value of 0.95. The learning rate for epoch number \(n\) (\(n\geq 1\)) can be calculated from initial learning rate easily using the following formula:  
\[lr_{n}=lr_{0}\times\gamma^{n-1} \tag{2}\]  
Using larger gamma values (0.95 to 0.99) results in high learning rates in the first 100 epochs which in turn results in larger gradient steps that reduce the training robustness of the model. Using small values of gamma increases the training time as well as reduces the chance to escape a local optimum in the loss function.