# DIFFBAS: An Advanced Binaural Audio Synthesis Model Focusing on Binaural Differences Recovery
## 1 Introduction
Input mono signal from source \(\mathbf{x}_{source}(t)\) is filtered by HRTF \(h_{trif}^{L/R}(t)\) and room impulse response \(h_{ri}^{room}(t)\) respectively and then superimposed with environmental noise \(\omega_{noise}(t)\) to produce the binaural signals \(\hat{\mathbf{y}}_{L}(t)\) and \(\hat{\mathbf{y}}_{R}(t)\); **(b)** The DNN-based BAS method directly models the propagation process from the sound source to the left and right ears to produce \(\hat{\mathbf{y}}_{L}(t)\) and \(\hat{\mathbf{y}}_{R}(t)\); **(c)** The framework of proposed DIFFBAS model. to left/right ears). They also suggested using the generative model DDPM (Denoising Diffusion Probabilistic Model) [18] to enhance the naturalness of synthesized audio. However, they jointly trained these two stages, and the model may still cause overfitting in the common path modeling part, thus affecting the accuracy of the specific path modeling. In fact, the spatial sense of human hearing mainly lies in the binaural differences perceived by the listeners [19] which is closely related to the specific path modeling instead of common path modeling. The overfitting problem in common path modeling may favor simulations of early attenuation and reverberation establishment, thus affecting the modeling of binaural differences and deviating from the original intention of the BAS task. In view of this, we propose the DIFFBAS method to decompose existing DNN-based methods (see Figure 1c). The design of DIFFBAS is intended to eliminate the modeling of common paths in the sound propagation process (such as reverberation effects), and focus on the modeling of specific paths, especially the filtering effects of the head and ears. This method not only reduces dependence on the dataset but also more accurately reproduces differences between ears. Specifically, DIFFBAS takes the mono audio at the point of the front head as input and adopts the newly proposed interaural phase difference (IPD) loss to supervise the model training. Such an operation is also reasonable in practice: the common path model can be trained separately and output a decayed mono signal which can be used as the input of DIFFBAS. Based on the decayed mono signals, DIFFBAS only has to focus on modeling the specific path to generate more accurate binaural audio which contains better spatial information. Meanwhile, the addition of IPD loss can better supervise the modeling of binaural differences based on the original loss supervision. Our contributions are summarized as follows:  
* A novel DNN-based BAS method, DIFFBAS is proposed. By focusing on modeling the specific path, DIFFBAS can avoid the overfitting problem caused by environmental and sound source diversity. The model employs new loss supervision for supplementary supervision and consequently improves the accuracy of synthesized binaural audio referring to the interaural phase difference between two ears. * To better train the DIFFBAS model, a new loss, namely IPD loss, is proposed. This loss fits the perceptual characteristics of the human ear and ensures that the model will not lose too much spatial information during the training process. In addition, IPD is regarded as a new evaluation metric that reflects the accuracy of binaural phase difference in synthesized audio. It is the first objective metric that can quantify the spatialization degree of binaural audio. * The performance of DIFFBAS has been validated on the only binaural dataset captured in the real world [15]. Sufficient contrast and ablation experimental results further demonstrate the effectiveness of DIFFBAS and IPD loss. The source code and the datasets are available at [[https://github.com/tongjiRain/DIFFBAS](https://github.com/tongjiRain/DIFFBAS)]([https://github.com/tongjiRain/DIFFBAS](https://github.com/tongjiRain/DIFFBAS)).