# Employing Huber and TAP Losses to Improve Inter-SubNet in Speech Enhancement
###### Abstract
The performance of a deep neural network for SE is influenced by various factors, including the hyperparameters, architecture, initialization, training data, and loss function. By carefully optimizing these elements, researchers can improve existing deep-neural-network-based SE methods. While modifying the network's architecture can be challenging, adjusting the loss function often yields more tangible results. Since the loss function is only utilized during the training phase and not during inference, this approach allows for performance enhancements without adding to the computational burden or time required for inference. Among various state-of-the-art DNN-based SE models, Inter-SubNet [25] is a masking-based and frequency-domain method since it pursues a complex ideal ratio mask (cIRM) [22] that applies to the STFT of the input signal. Essentially, Inter-SubNet is a refined and stream-lined version of FullSubNet [26] and FullSubNet+ [27], two highly effective SE models. While achieving superior results, Inter-SubNet boasts a significantly smaller model size, making it more computationally efficient. According to [25], Inter-SubNet was also rigorously tested on the DNS Challenge dataset and consistently outperformed some other leading and state-of-the-art methods, including DCCRN-E [14], Conv-TasNet [28], PoCoNet [29], DCCRN+ [30], TRU-Net [31], CTS-Net [32], FullSubNet [26], and FullSubNet+ [27]. Building upon the insights gained above, this study proposes a novel modification to Inter-SubNet, focusing on its loss function. By refining the loss function, we aim to enhance Inter-SubNet's performance without compromising its computational efficiency. The original Inter-SubNet model employs the mean-squared error (MSE) of cIRM as its loss function, which, while effective in certain scenarios, can exhibit limitations. The following may be included in these limitations:MSE is sensitive to outliers and can suffer from gradient explosion issues, which can impede the training process. * By only minimizing the estimation error of the cIRM, the MSE loss function overlooks the inherent variability in the importance of different time-frequency bins of a speech signal. This is important because human perception is not uniform across all frequency bands. * In scenarios with low signal-to-noise ratios (SNRs), MSE can cause the model to excessively reduce certain frequency regions where noise dominates, negatively impacting the quality of the speech component. * While human auditory perception operates on a non-linear scale, the cIRM is defined on a linear frequency scale. This discrepancy may further impede the efficacy of MSE in revising cIRM to capture perceptual details. To address these limitations, we suggest integrating Huber loss [33] and temporal acoustic parameter (TAP) loss [34] into the training of Inter-SubNet. Huber loss, which combines L1 and L2 losses, is less sensitive to outliers than MSE and is a robust loss function. On the other hand, TAP loss provides a detailed estimation for 25 low-level acoustic descriptors, offering a comprehensive evaluation of speech signal distortion. These modifications to Inter-SubNet are expected to improve speech enhancement performance by addressing gradient explosion issues and considering perceptual characteristics. It is important to note that these modifications only impact the training process and do not increase computational complexity or latency during inference. The experiments conducted on the VoiceBank-DEMAND database exhibit the superiority of the presented modifications. Our enhanced Inter-SubNet consistently outperforms the original model across various speech enhancement metrics, highlighting the effectiveness of our approach. The remainder of this study is organized as follows: An overview of the Inter-SubNet framework is provided in Section 2. Huber loss, TAP loss, and potential methods of integrating them with the initial MSE loss are introduced in Section 3. Section 4 provides the experimental setup, and Section 5 presents the experimental results and analyses.