# Employing Huber and TAP Losses to Improve Inter-SubNet in Speech Enhancement
## 2 Inter-SubNet  
This section briefly introduces the SE framework, Inter-SubNet, which serves as the model for further improvement. The flowchart of the training of Inter-SubNet is shown in Figure 1, and is described as follows:  
1. Initially, the input signal \(x[n]\) is transformed into a complex-valued spectrogram \(\mathbf{X}=\mathbf{X}^{r}+j\mathbf{X}^{i}\in\mathbb{R}^{F\times T}\) using the short-time Fourier transform (STFT). Here, \(F\) and \(T\) represent the total number of frequency bins and frames, respectively. The magnitude part of the spectrogram, denoted as \(\mathbf{X}^{m}\), is obtained by calculating the absolute value of each entry in the complex spectrogram. \(\mathbf{X}^{m}\) is then split into sub-band units \(\{\mathbf{b}_{f}\}_{f=1}^{F}\) using an \"unfold\" process. Each unit \(\mathbf{b}_{f}\) is generated by concatenating a frequency bin vector \(\mathbf{X}^{m}_{f}\in\mathbb{R}^{T}\) with its adjacent frequency bin vectors, extending \(N\) frequency bins on each side. That is, \[\mathbf{b}_{f}=\left[\mathbf{X}^{m}_{f-N};\cdots;\mathbf{X}^{m}_{f};\cdots; \mathbf{X}^{m}_{f+N}\right]\in\mathbb{R}^{(2N+1)\times T}.\] (1) Note that circular Fourier frequencies are used for those frequency indices \(f\) for \(\mathbf{X}^{m}_{f}\) if \(f>F-1\) or \(f<0\). 2. The sub-band units \(\{\mathbf{b}_{f}\}_{f=1}^{F}\) pass through a stack of two SubInter-LSTM (SIL) blocks and a fully connected layer to produce the complex ideal ratio mask (cIRM), \(\mathbf{M}^{r}+j\mathbf{M}^{i}\). Each SIL block comprises a SubInter module, an LSTM network, and a group normalization layer (G-Norm). The SubInter module plays a crucial role in capturing local and global spectral information, the flowchart of which is depicted in Figure 2. This module takes the sub-band units \(\{\mathbf{b}_{f}\}_{f=1}^{F}\) as input and outputs enhanced sub-bands \(\{\mathbf{\hat{b}}_{f}\}_{f=1}^{F}\) through a series of procedures: 1. The dimension transformation (permutation) and a linear layer are applied to the input sub-band units: \[\{\mathbf{h}_{f}\}_{f=1}^{F}=\mathcal{F}\Big{(}\text{transform}((\{\mathbf{b}_{f }\}_{f=1}^{F})\Big{)},\] (2) where the output \(\{\mathbf{h}_{f}\}_{f=1}^{F}\) is the first hidden representation that contains parallel sub-band information, and \(\mathcal{F}\) refers to a linear layer operation, and \"transform\((. )\)\" refers to the dimension transformation operation. 2. The first hidden representation is averaged across all sub-bands and then passed through another linear layer: \[\mathbf{\tilde{h}}=\mathcal{R}\Bigg{(}\frac{1}{F}\sum_{f=1}^{F}\mathbf{h}_{f} \Bigg{)},\] (3) where \(\mathcal{R}\) denotes a linear layer. The output \(\mathbf{\tilde{h}}\), the second hidden representation, contains the global spectral information due to the average operation considering cross-band dependencies and refers to sub-band interaction. 3. Each of the sub-bands \(\mathbf{h}_{f}\) is concatenated with \(\mathbf{\tilde{h}}\). Then, the set of all updated sub-bands undergoes the third linear layer and the dimension transformation (permutation). Additionally, a residual connection is implemented to add back the original units \(\{\mathbf{b}_{f}\}_{f=1}^{F}\). Therefore, the ultimate output for this module is as follows: \[\{\mathbf{\hat{b}}_{f}\}_{i=1}^{F}=\{\mathbf{b}_{f}\}_{i=1}^{F}+\text{ transform}\Big{(}\mathcal{P}(\{[\mathbf{h}_{f},\mathbf{\tilde{h}}]\}_{f=1}^{F}) \Big{)},\] (4) where \(\mathcal{P}\) denotes a linear layer. Obviously, the final enhanced sub-band features \(\{\mathbf{\hat{b}}_{f}\}_{f=1}^{F}\) intertwine global and local (sub-band) spectral information. Upon completion of the SubInter module, the LSTM network acquires the local and complementary global spectral information embedded within these sub-band features \(\{\mathbf{\hat{b}}_{f}\}_{f=1}^{F}\), which acts like the LSTM layers of the sub-band model in FullSubNet+ [27]. A G-norm layer is then used to normalize the LSTM outputs. 3.