# Upmix B-Format Ambisonic Room Impulse Responses Using a Generative Model
## 3 Upmixing Ambisonics Sound Field
### HiFi-Gan
The loss function for the discriminators in the network model includes the loss of the generative adversarial network \(\mathcal{L}_{adv}\), the loss of the multi-scale discriminator \(\mathcal{L}_{MSD}\), and the loss of the STFT discriminator \(\mathcal{L}_{STFT}\). The overall loss for the discriminator is defined as  
\[\mathcal{L}_{D}=\mathcal{L}_{adv}(D;G)+\gamma_{S}\mathcal{L}_{MSD}+\gamma_{P} \mathcal{L}_{STFT}, \tag{15}\]  
where \(\gamma_{S}\) and \(\gamma_{P}\) are weighting factors that are determined by hyperparameter search. The loss function \(\mathcal{L}_{adv}(D;G)\) is denoted as  
\[\mathcal{L}_{adv}(D;G)=\mathbb{E}[(D(\mathbf{p})-1)^{2}+(D(\mathbf{p}_{u})^{2 }]. \tag{16}\]  
The output of the \(D\) is between 0 and 1, and the training objective is to make \(D(\mathbf{p})\) approach 1 and \(D(\mathbf{p}_{u})\) approach 0. The loss of MSD is  
\[\mathcal{L}_{MSD}(\mathbf{p}_{u},\mathbf{p})=\|\mathbf{p}-\mathbf{p}_{u}\|_{1}, \tag{17}\]  
where \(\|\cdot\|_{1}\) represents \(l_{1}\)-norm, and \(\mathbf{p}\) and \(\mathbf{p}_{u}\) represent the measured higher-order microphone array signals and the generated signals, respectively. The loss function of the STFT discriminator is computed as the sum of the spectral convergence \(\mathcal{L}_{SC}\) and the spectral log-magnitude \(\mathcal{L}_{SD}\) over \(R\) different STFT resolutions as proposed in [27]. The STFT discriminator loss \(\mathcal{L}_{STFT}\) is defined as  
Figure 4: The Generator G architecture that is used to upmix the low-order Ambisonics signal \(\hat{\mathbf{p}}\) to \(\mathbf{p}_{u}\). \[\mathcal{L}_{STFT}(\mathbf{p}_{u},\mathbf{p}) =\sum_{r=1}^{R}[\mathcal{L}_{\mathcal{S}\mathcal{C}_{r}}(\mathbf{p} _{u},\mathbf{p})+\mathcal{L}_{\mathcal{SD}_{r}}(\mathbf{p}_{u},\mathbf{p})] \tag{18}\] \[=\sum_{r=1}^{R}\bigg{[}\frac{\|\mathcal{F}(\mathbf{p})\|-| \mathcal{F}(\mathbf{p}_{u})|\|_{F}}{\||\mathcal{F}(\mathbf{p})\||_{F}}+\frac{ \|\log|\mathcal{F}(\mathbf{p})|-\log|\mathcal{F}(\mathbf{p}_{u})|\|_{F}}{N} \bigg{]}. \tag{19}\]  
Here, \(\|\cdot\|_{F}\) denotes the F-norm, and \(\mathcal{F}(\cdot)\) is the STFT operator which transforms the flattened time-domain multi-channel RIRs of dimension \((M\times K)\times 1\) into the spectrogram of dimension \(L\times(W\times M)\). \(M\) represents the number of RIR channels, \(K\) is the number of time samples of the impulse response, \(L\) is the number of frequency bins, and \(W\) is the number of frames in STFT. The multi-resolution loss in the STFT discriminator is computed by considering four distinct STFT resolutions: 1024, 2048, 512, and 256. The hop size is set to half of the STFT length, which allows for overlapping segments during the transformation. We used a Hann window as the window function for the STFT. The loss function of the generator in HiFi-GAN is written as  
\[\mathcal{L}_{G}=\mathcal{L}_{adv}(G;D), \tag{20}\]  
where the loss \(\mathcal{L}_{adv}(G;D)\) of the generative adversarial network is  
\[\mathcal{L}_{adv}(G;D)=\mathbb{E}[(D(\mathbf{p}_{u})-1)^{2}]. \tag{21}\]  
During training, the generator network model is designed so that it produces signals that the four discriminators cannot distinguish from real signals. This encourages the generator to upmix the first-order ARIRs to higher orders. The Adam optimizer is employed to train the model, while the batch size is set to 8, and the training process iterates for a total of 65 epochs to yield the results.