# Employing Huber and TAP Losses to Improve Inter-SubNet in Speech Enhancement
###### Abstract  
In this study, improvements are made to Inter-SubNet, a state-of-the-art speech enhancement method. Inter-SubNet is a single-channel speech enhancement framework that enhances the sub-band spectral model by integrating global spectral information, such as cross-band relationships and patterns. Despite the success of Inter-SubNet, one crucial aspect probably overlooked by Inter-SubNet is the unequal perceptual weighting of different spectral regions by the human ear, as it employs MSE as its loss function. In addition, MSE loss has a potential convergence concern for model learning due to gradient explosion. Hence, we propose further enhancing Inter-SubNet by either integrating perceptual loss with MSE loss or modifying MSE loss directly in the learning process. Among various types of perceptual loss, we adopt the temporal acoustic parameter (TAP) loss, which provides detailed estimation for low-level acoustic descriptors, thereby offering a comprehensive evaluation of speech signal distortion. In addition, we leverage Huber loss, a combination of L1 and L2 (MSE) loss, to avoid the potential convergence issue for the training of Inter-SubNet. By the evaluation conducted on the VoiceBank-DEMAND database and task, we see that Inter-SubNet with the modified loss function reveals improvements in speech enhancement performance. Specifically, replacing MSE loss with Huber loss results in increases of 0.057 and 0.38 in WB-PESQ and SI-SDR metrics, respectively. Additionally, integrating TAP loss with MSE loss yields improvements of 0.115 and 0.196 in WB-PESQ and CSIG metrics. Inter-SubNet; Huber loss; TAP loss; speech enhancement +
Footnote †: journal: _Future Internet_  
signals. The work in [9] utilized CNNs for SE, showing that their model outperformed traditional SE methods. Recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, have also been employed for SE due to their ability to model temporal dependencies in sequential data. For example, ref. [10] introduced a time-domain SE model based on LSTM, demonstrating superior performance in real-time applications. Furthermore, recent advancements have led to end-to-end SE systems that directly map noisy audio to enhanced audio without the need for intermediate representations. Ref. [11] presented an end-to-end deep learning framework that performs multichannel speech enhancement in the time domain. In addition, incorporating attention mechanisms into DNN architectures has further enhanced the performance of SE models. The work in [12] introduced an attention-based model that dynamically focuses on relevant parts of the input signal, leading to improved noise suppression and speech quality. Moreover, to consider the demand for real-time SE, research towards optimizing DNN architectures has also been developed for efficiency. For example, ref. [13] proposed a lightweight network that maintains high performance while significantly reducing computational complexity, making it suitable for real-time applications. In addition to the network structure, an SE model can also be classified based on the following factors:  
* **The type of audio signal it operates on. ** For instance, time-domain methods [10; 14; 15] process the raw time-domain waveform of the noisy speech signal, while frequency-domain methods [16; 17; 18; 19] transform the noisy time-domain signal to the frequency domain using a short-time Fourier transform (STFT) and then learn a mapping from the noisy STFT representation to an enhanced STFT. * **The target that operates on the noisy representation to produce a cleaner output. ** For example, masking-based methods [20; 21; 22] estimate a mask or weight for each time-frequency bin of noisy speech to indicate the likelihood that the signal is speech rather than noise and then apply the mask to suppress noise. Conversely, mapping-based methods [14; 23; 24] directly map the noisy speech to a clean speech signal, with the DNN learning a complex mapping function to transform the noisy speech into a more speech-like signal.