# DIFFBAS: An Advanced Binaural Audio Synthesis Model Focusing on Binaural Differences Recovery
## 2 Methodology
### Framework of DIFFBAS  
As shown in Figure 2, DIFFBAS accepts the mono audio \(\tilde{\mathbf{x}}_{mono}\) and the relative position \(\mathbf{p}\) as the inputs, and warps the mono audio to produce two synthesized audio for the left and right ears. The backbone network of DIFFBAS can be any DNN. To illustrate, select two networks to implement DIFFBAS, including WarpNet [15] which encodes audio in the temporal domain and NFS [16] which encodes audio in Fourier space. The networks are trained under the supervision of a loss function composed of three terms.  
The first term is the \(\mathcal{L}_{2}\) which is defined as  
\[\mathcal{L}_{2}=\frac{1}{2T}\sum_{t=1}^{T}\left\{\left[\hat{\mathbf{y}}_{L}(t)-\bm {y}_{L}(t)\right]^{2}+\left[\hat{\mathbf{y}}_{R}(t)-\mathbf{y}_{R}(t)\right]^{2}\right\} \tag{3}\]  
where \(T\) is the audio length, \(\mathbf{y}_{L}\) and \(\mathbf{y}_{R}\) are the ground truth signals from left and right channels. \(\hat{\mathbf{y}}_{L}\) and \(\hat{\mathbf{y}}_{R}\) are the synthesized binaural audio. \(\mathcal{L}_{2}\) guarantees the synthesized left and right ear audio is close to the ground truth in the overall waveform.  
The second term \(\mathcal{L}_{pls}\) is the phase spectrum difference between the synthesized audios and ground truth, which is defined as follows  
\[\mathcal{L}_{pls}=\frac{1}{2F_{s}\cdot T_{s}}\sum_{w=0}^{F_{s}}\sum_{t=0}^{T_{ s}}\left(\left|\triangle\hat{\mathbf{Y}}_{L}-\angle\mathbf{Y}_{L}\right|+\left| \triangle\hat{\mathbf{Y}}_{R}-\angle\mathbf{Y}_{R}\right|\right) \tag{4}\]  
where \(\mathbf{Y}_{L}\) and \(\mathbf{Y}_{R}\) are the results of Short Term Fourier Transform (STFT) on \(\mathbf{y}_{L}\) and \(\mathbf{y}_{R}\), \(\hat{\mathbf{Y}}_{L}\) and \(\hat{\mathbf{Y}}_{R}\) are the results of STFT on \(\hat{\mathbf{y}}_{L}\) and \(\hat{\mathbf{y}}_{R}\), and \(F_{s}\) and \(T_{s}\) are the frequency and time resolutions of STFT. \(\angle\) means performing phase calculations on the spectrogram to obtain the phase spectrum. \(\mathcal{L}_{pls}\) helps to better model sound time delay.  
Figure 2: The overall framework of DIFFBAS. Our method takes mixed monaural audio (the mean value of binaural audio) and sound source location information as input. We adopt a joint loss for supervision, where the IPD is designed to enhance the recovery of interaural differences. We apply our method on two open-source backbone networks, including WarpNet [15], which encodes position information as time-domain convolution weights, and NFS [16], which encodes position information as frequency-domain masks.  
The third term is a newly proposed loss, namely \(\mathcal{L}_{IPD}\), which is used to supervise model training from the perspective of the interaural phase difference to retain better spatial information. It has been explained in more detail in Sections 2.3 and 2.4.  
We want to stress that during the training process, the input of DIFFBAS \(\mathbf{x}_{mono}\) is approximated by the average signals of the collected signals from the left and right ears, i.e., \(\mathbf{x}_{mono}=\frac{\mathbf{y}_{L}(n)+\mathbf{y}_{R}(n)}{2}\). Such approximation is reasonable, because [PERSON] et al. [17] pointed out that the signals collected at the front of the head can be represented by the average signals from the left and right ear channels. In addition, [PERSON] et al. [20] also used the average signals from the left and right channels as the approximation of the mono audio collected near the head. Experimental results have demonstrated that by replacing the training input signals and incorporating the new IPD loss term, the synthesized binaural audio exhibits more accurate spatial cues to the listener.