# Two-Stage Single-Channel Speech Enhancement with Multi-Frame Filtering
## 3 Method  
### Framework and Network Architecture  
The proposed system is illustrated in Figure 1, with the objective of removing background noise while minimizing distortion, a more complex problem than simple noise removal. To tackle this challenge, we adopted a problem decomposition approach and integrated two neural networks, DNN\({}_{1}\) and DNN\({}_{2}\). DNN\({}_{1}\) aims to suppress noise components coarsely, while DNN\({}_{2}\) is responsible for further noise removal and minimizing distortion introduced during enhancement. To this end, we adopt the following two measures. Firstly, we introduced a filter module that calculates a low-distortion filter based on the enhancement result of DNN\({}_{1}\) (\(\hat{X}^{(1)}\)), and applies it to the noisy spectrum (\(Y\)). The filtered output (\(\hat{X}^{(F)}\)) is then concatenated with \(\hat{X}^{(1)}\) and fed into DNN\({}_{2}\) as extra features. The purpose of this approach is two-fold. First, since the single-channel signal has no spatial information that can be  
Figure 1: The framework of the proposed system. exploited, we hope the result of multi-frame filters can convey the interframe correlation to DNN\({}_{2}\). Second, we wish the low-distortion filtered result can provide information that is complementary to the result of DNN\({}_{1}\), thus helping DNN\({}_{2}\) to produce less distorted speech, which is benifical to downstream tasks such as ASR. Secondly, we concatenate \(Y\) with \(\hat{X}^{(1)}\) and \(\hat{X}^{(F)}\) as the input for DNN\({}_{2}\), inspired by [6]. Through theoretical analysis and experiments in [6], it was shown that adding a scaled version of the noisy signal to the enhanced signal can monotonically increase the signal-to-artifact ratio under mild conditions and improve ASR performance. Thus, we believe that concatenating \(Y\) with \(\hat{X}^{(1)}\) and \(\hat{X}^{(F)}\) can provide essential information for DNN\({}_{2}\) to reduce distortion during the training process. We employ the TCN-DenseUNet described in [8] and modify it into a single-channel version for DNN\({}_{1}\) and DNN\({}_{2}\). Although both DNN\({}_{1}\) and DNN\({}_{2}\) adopt the TCN-DenseUNet structure, it should be noted that they do not share parameters. The main reason is that DNN\({}_{1}\) and DNN\({}_{2}\) focus on different tasks, while DNN\({}_{1}\) is concerned with removing background noise as much as possible, DNN\({}_{2}\) aims to make the most of the filter output and noisy speech to minimize distortion. We believe that not sharing parameters between the two networks can lead to better performance, which is supported to some extent by the experimental results in Section 5.4. TCN-DenseUNet is a variant of U-Net, with a temporal convolutional network (TCN) network inserted between the encoder and decoder. The DenseNet blocks are also inserted between different layers of the encoder and decoder of the U-Net. Figure 2 shows the diagram of the TCN-DenseUNet. The encoder contains a 2D convolution layer and seven convolutional blocks, while the decoder contains seven deconvolutional blocks and a 2D deconvolution layer. Skip connections are added between the encoder and decoder. Each convolutional block consists of a 2D convolution layer, an exponential linear units (ELU) nonlinearity layer, and an instance normalization (IN) layer. The deconvolutional block has the same structure as the convolutional block, except that the 2D convolution layer is replaced with the 2D deconvolution layer. The DenseNet blocks consist of five convolutional blocks and the TCN network contains four layers, each with seven dilated convolutional blocks. Figure 2: The diagram of TCN-DenseUNet. The detailed setup for the TCN-DenseUNet is also shown in Figure 2. DenseNet block is represented by DenseBlock\((g_{1},g_{2})\), where \(g_{1}\) and \(g_{2}\) are the growth rates for the first four and the last convolutional block, respectively.