## 5 Conclusions  
In this paper, a personalized HRTF estimation method has been proposed on the basis of deep neural networks using anthropometric measurements and ear images. In particular, while a conventional DNN-based method aimed to estimate HRTFs using the anthropometric data including head, torso, and pinna measurements, the proposed method replaced pinna measurements with ear images due to the difficulty of obtaining pinna measurements for live human ears. Thus, the neural network in the proposed method was composed of three sub-networks. The first one was a DNN to represent the head and torso measurements and the second one was a CNN for extracting pinna measurements from edge-detected ear images instead of actually measured pinna data. The two sub-networks were then merged into another DNN to estimate a personalized HRTF. The performance of the proposed personalized HRTF estimation method was evaluated in terms of both objective and subjective tests. For the objective tests, the RMSEs and LSDs between the reference \begin{table}
\begin{tabular}{c c c c} \hline
**Azimuth** & **CNN-DNN HRTF** & **CNN-DNN HRTF with ZCA** & **CNN-DNN HRTF with ZCA + Image Shift** \\ \hline \(-135^{\circ}\) & \(-20.26\) & \(-20.20\) & \(-19.44\) \\ \(-80^{\circ}\) & \(-18.60\) & \(-18.60\) & \(-18.49\) \\ \(-45^{\circ}\) & \(-16.35\) & \(-16.16\) & \(-15.93\) \\ Avg. & \(-18.40\) & \(-18.32\) & \(-17.95\) \\ \hline \end{tabular}
\end{table} 
Table 6: Comparison of average RMSEs of the proposed HRTF estimation method with/without data augmentation at \(-135^{\circ}\), \(-80^{\circ}\), and \(-45^{\circ}\). \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Average HRTF** & **DNN(17) HRTF** & **CNN–DNN HRTF** \\ \hline Accuracy within \(\pm 15^{\circ}\) (\%) & 70.0 & 70.0 & 80.0 \\ Accuracy within \(\pm 30^{\circ}\) (\%) & 72.5 & 82.5 & 85.0 \\ Front/back confusion rate (\%) & 25.0 & 15.0 & 12.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of average accuracies within specific margins (\(15^{\circ}\) and \(30^{\circ}\)) and front/back confusion rate for different HRTF estimation methods. and estimated HRTFs were measured. For the subjective test, a sound localization experiment was performed, and the distance perceived by listeners after applying the estimated HRTF to a sound source was measured. After that, the performance of the proposed method was compared with those of HRTF estimation methods using average HRTF, the estimated HRTF by a DNN trained with anthropometric measurements, and the estimated HRTF by a DNN trained with only head and torso measurements. Consequently, it was shown from the objective evaluation that the proposed method decreased the RMSE and LSD by 0.02 and 0.85 dB, respectively, compared to the DNN-based method using anthropometric data without pinna measurements. In addition, it was shown form the subjective evaluation that the proposed method provided higher localization accuracy of 6% than the DNN-based method. In addition, the front/back confusion rate for the proposed method was reduced by 2.5% compared to the DNN-based method. Next, data augmentation was performed to increase the training data by applying zero-phase component analysis (ZCA) and image shifting. Thus, it was shown that after data augmentation, the proposed method could increase the RMSE but reduce the LSD by 0.29 dB, compared to only using ear images from the CIPIC HRTF database. In future work, to improve the performance of the proposed CNN-DNN HRTF estimation method, different model structures need to be studied, such as a residual network [37] or a dense network [38]. Moreover, the CIPIC HRTF database is too small to train deep learning models. Even though data augmentation has been performed in this paper, further sophisticated investigation of the effect of data augmentation on the performance of the proposed method, particularly the RMSE, will be studied. All authors discussed the contents of the manuscript. H.K.K. contributed to the research idea and the framework of this study, and G.W.L.