# Low-Order Spherical Harmonic HRTF Restoration Using a Neural Network Approach
## 1 Introduction  
Virtual reality (VR) and augmented reality (AR) technologies are on the rise, through the advent of commercially available and affordable VR/AR headsets, with applications in gaming, education, therapy, social media and digital culture, amongst others. To provide a high fidelity immersive experience in virtual environments requires good quality spatial audio. To achieve this, the VR/AR technology must be able to deliver to the ears the same binaural cues as would be experienced in real life [1, 2]. These are Interaural Time Difference (ITD), Interaural Level Difference (ILD) and spectral cues introduced by the ear pinnae and torso [3, 4]. Head related transfer functions (HRTFs) are sets of binaural filters that encapsulate these cues from different angles relative to the head in three dimensions. Sound sources can be spatialised by direct convolution with the a given HRTF pair representing the intended sound source direction. Alternatively, a virtual loudspeaker framework can be employed, wherein methods such as vector base amplitude panning (VBAP) [5] or Ambisonics [6] are used to render sources between virtual loudspeaker points formed from the HRTFs [7]. Both methods typically require a high number of HRTF measurements to ensure good spatial resolution in the rendered audio [8].  
However, HRTFs are highly personalised due to different head and ear shapes. Using mismatched HRTFs can affect timbre quality, localisation performance and externalisation. Currently, the main way to obtain personalised HRTFs is through physical measurements, where microphones are placed at the ear canal of a subject and the loudspeakers are positioned at different angles relative to the head to measure the transfer functions [9, 10]. This measurement process is often tedious and requires substantial setup and calibration. Recent developments have been made in HRTF-based selectionbased on anthropomorphic data extracted from photographs of the ear [11] and HRTF simulation using 3D head models [12]. However, simulations are computationally expensive and usually require a lot of processing time [13,14].  
To simplify the measurement process, different HRTF interpolation methods have been proposed to acquire dense HRTF sets from sparse HRTF measurements. The current state of the art method is spherical harmonic (SH) interpolation, which leverages the spatial continuity in the spherical harmonic (SH) domain and uses it as a bridge to spatially up-sample a sparse HRTF measurement set to a denser one [15,16]. Depending on the number of sparse HRTF measurement points and SH order, important high frequency information can be lost in this process. Consequently, listeners may perceive timbre difference and weakened localisation performance in practical use.  
Recently, developments in machine learning have shown great improvement in neural style transfer and data restoration, especially in the image domain [17,18]. This paper investigates whether similar models can be used to restore the distorted high frequency data in SH-interpolated HRTFs.  
This paper is organised as follows: Section 2 covers relevant background information on SH HRTF interpolation. It also discusses the motivation for using a machine learning approach along with identification of some candidate models. Section 3 discusses the method used in this study, including the data pre-processing workflow, a baseline model and different techniques investigated on top of the baseline model. Section 4 describes the evaluation of the performance of the model based on perceptual spectral difference and localisation performance. Section 5 discusses the results and potential directions for the work. Section 6 concludes the paper.