# Low-Order Spherical Harmonic HRTF Restoration Using a Neural Network Approach
## 4 Evaluation
### Localisation Performance  
The localisation performance was analysed with [PERSON]'s model and [PERSON]'s model in the Auditory Modelling Toolbox (AMT) [68; 69; 70]. [PERSON]'s model is for frontal azimuth localisation on the horizontal plane and [PERSON]'s model is for the frontal sagittal plane. Table 8 shows the localisation results of the proposed model. The frontal azimuth localisation mean error from May's model shows improvement in all HRTF datasets. However, for frontal sagittal plane with [PERSON]'s model, results in RMS error and quadrant errors indicate all the reconstructed HRTFs perform worse in the frontal sagittal plane localisation. The current model suffered in localisation tasks, perhaps because it used smooth L1 loss as the loss function. The smooth L1 loss only focuses on the magnitude difference at each frequency point. There may not be any meaningful connection between these magnitude differences and localisation performance. Therefore, it is no surprise that the model failed to optimised the HRTF restoration for the localisation performance. A future model could add some types of localisation error into the loss function to improve the localisation results. and more measurements should have better performance than the current one, as less data needs to be restored. The results showed that a simple ML model can be used to restore distorted SH-interpolated HRTFs, although the current state of this model is far from optimised for application. It is believed that there will be significant improvements if more HRTF measurements are available for training in the future. Under the current situation, one way to improve the model is through hyper-parameter tuning, including the parameters for regularisation. An alternative method that may reduce over-fitting is data augmentation. Currently, HRTF measurements are expensive and tedious and therefore it is not very likely that there will be a huge increase in HRTF measurement data from standard methods in the near future. To augment the current dataset, one possible way is to use more different sparse HRTF configurations or a different SH order to train the model. Table 6 shows that even if the extra data are not perfect, it is possible that it can still improve the model's performance in some cases. Similar to data augmentation, noise injection is a different regularisation method that has be shown to work better than weight decay in some cases [71; 72; 73]. By picking the right parameters, it is believed that it could generalise better across measurement methods, as the model could focus on the general information across various HRTF measurements as opposed to the artefacts introduced by different measurement methods. Another problem that was observed from the current model was the localisation performance. Although it shows some improvement in the horizontal plane, the sagittal error needs further improvement. As discussed in Section 4, it is believed that the smooth L1 loss function only compares the difference at each frequency and fails to capture other useful metrics of HRTFs. Potentially, some custom loss functions could be implemented to improve the model. [PERSON] et al. [74] trained a separate machine learning model for content loss and implemented a special function for style loss. Similar methods such as training a localisation model as a localisation loss function may be able to solve the problem. Furthermore, with the recent success in generative adversarial networks (GAN), it should be possible to build a GAN based on localisation performance [37; 75; 76; 77]. However, as a GAN can be unstable to train and it usually requires a lot of tuning, it may not be the most effective way for SH-interpolated HRTF restoration. This paper has shown general insights of using machine learning for HRTF reconstruction. According to Tables 7 and 8, and the discussion in Section 4, to apply the idea to real-life applications, optimising the model for some narrative tasks should yield better performance.