# Toward Personal Identification Using Multi-Angle-Captured Ear Images: A Feasibility Study
## 4 Distributions of the Training and Testing Data
To do this, we adopted Grad-CAM (a method for visualizing the rationale behind CNN-based image identification). Figure 4 shows the outline of Grad-CAM. Grad-CAM presents the contribution to image identification as a heatmap based on the following four steps (corresponding to 1-4 in Figure 4):

1. It obtains the class C prediction of the input image. 2. It performs backpropagation based on the output probability of the class C prediction and calculates the gradient with respect to the feature map of the final convolutional layer. 3. It averages the calculated gradients for each feature map and uses them as weights for each feature map. 4. It linearly combines the feature maps of the final convolutional layer using the weights obtained in Step 3, followed by applying the ReLU function to generate the heatmap. Algorithm 2 shows the pseudocode for generating heatmaps that visualize the identification rationale of the input ear images based on Grad-CAM. ```
Input:_Ear Image Model \(s\) : The number of layers from the input layer to the final convolutional layer \(p\) : The size of one side (in pixels) of the ear image(square) Output:HeatMapEarImage
1:main
2:\(\textit{Model\_Output}\leftarrow\textbf{FIT}(\text{Model},\textit{EarImage})\) #Input ear images into the _Model_ and obtain a 10-class pribabilith distribution
3:\(C\leftarrow\textbf{ARGMAX}(\textit{Model\_Output})\) #Obtain the prediction class \(C\) (the class with the maximum probability) from the \(\textit{Model\_Output}\) (10-class probabilith distribution)
4:\(C\_Score\leftarrow\textit{Model\_Output}[C]\) #Obtain the prediction score for the prediction class \(C\)
5:\(\textit{LastFeatureMaps}\leftarrow\textbf{GETFEATMAP}(\textit{Model},\text{s})\) #Obtain the feature maps of the final convolutional layer
6:\(\textit{Gradients}\leftarrow\textbf{COMPUTE\_GRAD}(C\_Score,\textit{LastFeatureMaps})\) #Compute the gradients of the _LastFeatureMaps_ with respect to the score of the prediction class \(C\)
7:\(\textit{Weights}\leftarrow\textbf{AVERAGE}(\textit{Gradients})\) #Compute the _Weights_ for each feature map by averaging the _Gradients_ of the feature maps
8:\(\textit{Cam}\leftarrow\textbf{LINEAR\_COMBI}(\textbf{Weights},\textit{LastFeatureMaps})\) #By weighting each feature map (in _LastFeatureMaps_) with _Weights_ and Performing a linear combination, a matrix(\(\textit{Cam}\)) that retains the importance of each pixel is obtain
9:\(\textit{Cam}\leftarrow\textbf{RELU}(\textit{Cam})\) #Apply _ReLU_ function to remove negative values
10:\(\textit{CamResized}\leftarrow\textbf{RESIZED}(\textit{Cam},\textit{p})\) #Resize the _Cam_ to match the size of _EarImage_\(p\times p\)
11:\(\textit{Heatmap}\leftarrow\textbf{COLORMAP}(\textit{CamResized})\) #Generate the _Heatmap_
12:\(\textit{Heatmap}\leftarrow\textbf{BLEND}(\textit{EarImage},\textit{Heatmap})\) #Overlay the _Heatmap_ on the _EarImage_
13:returnHeatmapEarImage
```

**Algorithm 2**Pseudocode for generating ear heatmap images using Grad-CAM

Figure 4: Grad-CAM outline. ## 8 Results

### Results of the Identification Experiments

In this study, we implemented three different experimental settings to evaluate the identification performances for multidirectionally captured ear images. Table 4 presents the number of training and test images employed in each identification experiment as well as the identification results. Experiment setting (i) comprised an experiment in which the ear images captured from the same direction as that of the training data were employed as the test data. In this setting, scores exceeding 0.975 were achieved for all the evaluation metrics across all the experiments, indicating that the models performed the ear-based identification with high accuracy when the training and testing data were obtained from the same direction. Experiment setting (ii) comprised an experiment in which the ear images captured from a different direction from that of the training data were deployed as the test data.