# Upmix B-Format Ambisonic Room Impulse Responses Using a Generative Model
###### Abstract  
Ambisonic room impulse responses (ARIRs) are recorded to capture the spatial acoustic characteristics of specific rooms, with widespread applications in virtual and augmented reality. While the first-order Ambisonics (FOA) microphone array is commonly employed for three-dimensional (3D) room acoustics recording due to its easy accessibility, higher spatial resolution necessitates using higher-order Ambisonics (HOA) in applications such as binaural rendering and sound field reconstruction. This paper introduces a novel approach, leveraging generative models to upmix ARIRs. The evaluation results validate the model's effectiveness at upmixing first-order ARIRs to higher-order representations, surpassing the aliasing frequency limitations. Furthermore, the spectral errors observed in the Binaural Room Transfer Functions (BRTFs) indicate the potential benefits of using upmixed ARIRs for binaural rendering, significantly improving rendering accuracy. Ambisonics; upmix; generative model +
Footnote †: journal: _Applied Sciences_  
HOA microphone array requires deploying at least \((N+1)^{2}\) microphones on the surface of the sphere [10]. In practice, the FOA microphone array is a more effective solution and has been widely used for recording ARIRs. To further enhance the spatial representation capability, upmixing algorithms can be applied to extend low-order ARIRs recordings to higher-order ARIRs. Two notable methods are the Spatial Decomposition Method (SDM) [15] and the Ambisonics Spatial Decomposition Method (ASDM) [8]. These methods assume that there is a single dominant direction of arrival (DOA) within each time frame. They estimate this DOA and then re-encode the FOA signal into a HOA representation based on the estimated principal source direction. [PERSON] et al. [16] utilized three additional directions that depend on the detected DOA to represent the diffuse sound component, increasing the echo density of the estimated signal. However, these methods assume temporal sparsity and may not effectively model complex reverberation components containing multiple overlapping DOAs. Spatial Impulse Response Rendering (SIRR) was proposed to estimate narrow-band DOA and recreate the diffuse components of the captured sound field based on a diffuseness estimate [17; 18]. In summary, traditional methods for modeling ARIRs have primarily focused on synthesizing specific components, such as the direct path, early reflections, or diffuse components. These approaches typically involve first estimating the DOA of the recorded signals before synthesizing HOA signals. A different approach focuses on upmixing by augmenting the number of microphone signals on a rigid spherical array. These additional signals are subsequently transformed into Ambisonics using a spherical harmonic transform [19]. Further, [PERSON] et al. proposed using the Physics-Informed Neural Network (PINN) to estimate the sound field around a rigid sphere based on the measurement obtained from a set of sparsely distributed microphones mounted on the rigid sphere [20]. This method, however, focuses on modeling the sound field in the frequency domain. In recent years, generative models, particularly generative adversarial networks (GANs) [21], have gained significant attention and success in various domains, including audio super-resolution and RIR synthesis [22; 23; 24; 25]. The GAN framework consists of a generator and a discriminator. The generator synthesizes signals that resemble real data, while the discriminator determines the authenticity of the generated signals. These two models are trained iteratively, playing against each other. In the field of audio synthesis and speech enhancement, HiFi-GAN has emerged as an improved GAN approach [26]. In the study described in [27], HiFi-GAN was specifically employed to broaden the bandwidth of microphone measurements. In this work, we present the first end-to-end deep learning approach for upmixing FOA RIRs directly in the time domain using HiFi-GAN.