# Environmental Attention-Guided Branchy Neural Network for Speech Enhancement
## 1 Introduction  
Speech enhancement techniques have been widely used to cope with the noise interference problem in the front end of many speech applications, such as mobile phones, hearing aids, and speech recognition products. The early conventional methods paid more attention to the optimization of suppression gain function [1; 2; 3], the estimation of the noise spectrum [4; 5; 6], and prior signal-to-noise ratio (SNR) [7; 8]. In recent years, deep neural network (DNN)-based speech enhancement methods have shown significant performance advantages over the traditional approaches in complex noise environments, even the extremely nonstationary noises. Whether utilizing masking-based [9; 10; 11] or mapping-based [12; 13; 14; 15; 16; 17] DNN methods, their general rule is to optimize the loss function between the ideal and noisy targets to achieve as little error as possible in the global noisy speech dataset. Consequently, richer datasets, and a better objective function and neural network models were further explored to guarantee the robust generalization ability of DNN models to cope with the diversified noise environments in real life. In the research of [14], experimental results demonstrated that the richness of the clean speech samples and the noise samples were the two crucial aspects to improve the generalization capacity of DNNs. Recently, researchers have paid more attention to the optimization of DNN models for the speech enhancement task. Considering the temporal relationship of speech signals, long short-term memory (LSTM) networks [18] and temporal convolutional networks (TCN) [19] have been proven to have better speech reconstruction capacities for speech enhancement. In addition, the optimization of the objective function [20; 21] has also been exploredfor the performance improvement of DNN-based speech denoising. Generative adversarial networks (GAN) [22] have been another new solution to train the DNN model to learn noise suppression abilities. These methods aim to train a common DNN denoising model for all kinds of noise interference in different application scenarios. However, most speech products have their specific application scenarios. When the application scenario is determined, the types of noise interference are known. In different noise environments, the optimal parameters of speech enhancement algorithms are different. As investigated in [23], this prior environmental information was helpful to further improve the speech enhancement methods to achieve better noise suppression effect in specific noise environments. Therefore, some researchers started to design the speech enhancement algorithms by incorporating the prior noise information. In [24; 25; 26], the noise classification module was integrated into the Wiener filter and some statistical model-based speech estimators. Guided by the noise classification module for optimal parameter selection, the performance of traditional methods was further improved in prior known noisy environments. A similar idea was applied to the DNN-based denoising algorithms [27; 28; 29]. In the recent studies of [28; 29], under the guidance of the noise classification unit, several independent DNN models were trained to give full play to their respective noise reduction capabilities in different noise environments. Although this \"divide and conquer\" strategy effectively improves the noise reduction performance, it also increases the storage burden and reduces the complementarity between different noises. In this work, we proposed a novel branchy neural network (BNN) framework to improve the storage burden and noise complementarity problem of separate training. There are two key modules working together in our proposed framework, a classification network and a denoising network. First, the noise classification network is trained to analyze the presence probability of each noise component for every input noisy frame.