# Speech Enhancement Based on Unidirectional Interactive Noise Modeling Assistance
###### Abstract
In general, the CRN structure is commonly utilized in SE, and CRN-based models have exhibited excellent performance. Therefore, we also adopt the CRN structure as the network backbone in constructing the proposed DNN model. Moreover, many researchers have enhanced the SE technique from various other perspectives. CTS-Net [8] decouples the SE task into coarse magnitude spectrum estimation and fine-grained complex-valued spectrum refinement in a sequential manner, thus proposing a two-stage paradigm. Later, GaGNet [20] modifies the coarse and fine-grained estimation in CTS-Net from sequential to parallel processing, which prevents error accumulation and ensures hierarchical optimization toward the complex-valued spectrum. Meanwhile, some works implement SE using multi-domain processing techniques. For example, CompNet [21] enhances the noisy speech sequentially in both the time and TF domains, and FDFNet [22] processes the speech spectrum sequentially in the Fourier transform and discrete cosine transform domains. Moreover, several full-band and sub-band fusion SE models have been proposed to integrate local sub-band spectrum features with global cross-band dependencies for better performance, including FullSubNet [23], FullSubNet+ [24], and LFSFNet [25]. Based on these models, Inter-SubNet [26] discards the full-band model and utilizes a sub-band interaction method to supplement global spectrum patterns for the sub-band model. Furthermore, generative SE approaches, such as NASE [27], have also garnered increasing attention. All of the aforementioned methods take the clean speech as the predicting target, so they can be referred as the speech-prediction methods. However, these methods lack explicit analysis and modeling for the characteristics of noise components. In addition, predicting the clean speech from low-SNR signals can be challenging, potentially resulting in poor quality of the enhanced speech. Consequently, the noise-prediction method [28] attempts to predict the pure noise, which is then subtracted from the noisy speech to indirectly obtain the enhanced speech. But the benefit is still limited. Motivated by the speech-prediction and noise-prediction methods, some subsequent methods [29; 30; 31] propose a dual-branch network to simultaneously predict the target speech and noise signals. Additionally, the bidirectional information interaction module is typically introduced between the speech and noise branches to further optimize the speech and noise modeling process [29]. Although the interactive speech and noise modeling-based dual-branch network has illustrated excellent performance, it significantly increases the model complexity. Unfortunately, the huge model complexity may be intolerable in many resource-constrained devices. In this work, we propose a unidirectional information interaction-based dual-branch network, abbreviated as UniInterNet. In the UniInterNet, intermediate features only flow from the speech branch to noise branch, thereby promoting the noise modeling. While the information flow from the noise branch to speech branch is discarded. in this way, the speech branch does not depend on any information from the noise branch when generating the enhanced speech. Therefore, the noise branch is utilized to assist the learning of speech branch by backpropagating the noise modeling results during the model training stage, and it is not needed in the deployment phase. Compared to the previous dual-branch methods [29; 31], the proposed UniInterNet not only retains the benefit of noise prediction for SE but also reduces the deployment cost by avoiding the use of dual-branch network in the model inference stage. Moreover, previous methods typically employ the same network structure to construct the speech and noise branches. In contrast, we adopt a more complicated, and even non-causal network structure for the noise branch, while limiting the complexity and ensuring the causality of the speech branch.