## 3 Model  
### Structure  
For source separation, various deep networks have made significant progress, especially the recently proposed end-to-end time-domain separation networks. They follow one unified framework: encoder-separator-decoder. In this study, three typical and excellent networks in the field of speech separation are trained: Convolutional Time-domain Audio Separation Network (ConvTasNet) [15], Dual-Path Recurrent Neural Network (DPRNN) [22], and Dual-Path Transformer Network (DPTNet) [23]. They all reported state-of-the-art performances on many public speech separation datasets and have great potential towards the universal source separation problem. Although with comparable performance, there is a considerable gap in the network structures. ConvTasNet, DPRNN, and DPTNet are based on the three most commonly used architectures in the field of speech processing: Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Transformer, respectively. CNN [24] is one typical feed-forward architecture, which is especially suited for visual pattern recognition due to the properties of translation invariance and weight sharing. RNN [10] processes the input as a sequence based on recurrent connections, useful for learning temporal dynamics. Transformer [25] is a recently proposed architecture, which is one sequence transduction model based on multi-headed self-attention. It is more parallelizable and requires less training time than CNN and RNN. In addition, the introduction of the residual path, skip-connection path, and dual-path makes networks more complex and flexible, incorporating more differences. The details of the three networks are shown in Figure 1, where (a) is the framework of separation networks, and (b,c,d) represent the separator in ConvTasNet, DPRNN, and DPTNet, respectively. Figure 1: The structures of three networks, where (**a**) indicates the unified framework for source separation networks: encoder-separator-decoder, (**b–d**) indicate the separator in ConvTasNet [15], DPRNN [22], and DPTNet [23], respectively. The encoder in source separation systems is used to transform the time domain waveform into a non-negative 2-D intermediate representation (X), which is implemented by a 1-D convolutional layer and a rectified linear unit (ReLU) layer. The kernel size (also called window size) in the convolutional layer controls the frame rate of the mixture. It is an essential parameter that affects network performance and is also explored in the section on separation mechanisms. Then separator, the main body of the network, estimates masks for two sources (M\({}_{1}\) and M\({}_{2}\)). The estimated masks are multiplied with X to obtain 2-D representations of each source (Y\({}_{1}\) and Y\({}_{2}\)). Finally, the time-domain signal is reconstructed through the decoder, implemented by a 1-D transposed convolutional layer. The separator can be implemented through a variety of network structures. The separator in ConvTasNet is a fully convolutional network. At first, input is processed by a global layer normalization (LayerNorm) and a 1 \(\times\) 1 convolutional layer, indicating a 1-D convolution with a kernel size of 1. It determines the channels for the following modules and is regarded as the bottleneck layer. The main body of ConvTasNet contains \(R\) repeated modules (dashed box), and each module is composed of \(N\) stacked Dilated Conv-Blocks with exponentially increased dilation factors. Each Dilated Conv-Block is a cascade of 1 \(\times\) 1 convolution, PReLU, normalization, depth-wise dilated convolution, PReLU, normalization layer. Two 1 \(\times\) 1 convolutional layers serve as the residual path and the skip-connection path, where the output of the residual path is the input of the next block, and the skip-connection paths of all blocks are summed up and used as the input to a 1 \(\times\) 1 convolutional layer and a nonlinear activation layer to estimate two masks (M\({}_{1}\) and M\({}_{2}\)).