# Iteratively Refined Multi-Channel Speech Separation
## 2 Proposed Method
### Initial Separation  
In Figure 1, the structure of the initial separation is shown. It is composed of three parts, i.e., the TFDPRNN used for pre-separation with one input (mixed signal), the MVDR beamformer, and post-separation consisting of the same TFDPRNN with two inputs (mixed signal and the output of the MVDR beamformer). In this structure, the mixed signal from multiple microphones is first fed into this pre-separation network. Then, the pre-separated signal and the mixed speech signal are used to obtain the statistical information of the MVDR beamformer. Finally, the post-separation network is arranged at the backend of the beamformer to obtain the finally refined initially separated speech. The following is a detailed introduction to the initial separation. Figure 1: The overall structure of the initial separation. During the pre-separation, the signals collected by each microphone are individually fed into the respective TFDPRNN modules, where each network adopts a classical encoder-separator-decoder structure for the processing. 1. In the encoder section, firstly, the mixed signal \(x_{c}\) is transformed into the time-frequency representation \(Y_{c}\) by short-time Fourier transform (STFT). Then, this representation \(Y_{c}\) is applied to the dynamic range compression (DRC) module to obtain \(Y_{DRC}\). Subsequently, the local features \(Y_{Conv}\) are extracted from \(Y_{DRC}\) through a 2D convolutional (Conv2D) layer. Finally, these features \(Y_{Conv}\) are passed through a rectified linear unit (ReLU) activation function to obtain the encoded feature \(E_{c}\). The whole encoder section can be expressed as: \[E_{c}=\text{Encoder}\{x_{c}\}_{c}\] (2) where \(\text{Encoder}\{.\}_{c}\) corresponds to the encoder of the \(c^{\text{th}}\) microphone and \(E_{c}\) denotes the representation of encoder of the \(c^{\text{th}}\) microphone. 2. In the separator section, firstly, the encoded feature \(E_{c}\) is sent to the layer normalization (LN) for standardization, followed by a Conv2D layer to obtain the feature \(\hat{Y}_{conv}\). Subsequently, the feature \(\hat{Y}_{conv}\) is sent to \(N\) time-frequency domain scanning blocks using a time-frequency scanning mechanism [21; 22]. Each scanning block consists of two recurrent modules, where the first recurrent module utilizes a bi-directional LSTM (BLSTM) network layer along the frequency axis and the second recurrent module utilizes BLSTM along the time axis. Both modules include reshaping, LN, and fully connected (FC) operations. Finally, after processing through these modules, the features are further refined through a Conv2D layer and the ReLU activation function, resulting in the separated mask \(\hat{M}_{q,c}\). The whole separator section can be expressed as: \[\hat{M}_{q,c}=\text{Separator}\{E_{c}\}_{c}\] (3) where \(\text{Separator}\{.\}_{c}\) denotes the separator corresponding to the signal of the \(c^{\text{th}}\) microphone and \(\hat{M}_{q,c}\) denotes the mask of the \(q^{\text{th}}\) speaker in the \(c^{\text{th}}\) microphone. Thus, the separated masks are multiplied element-wise with the encoded feature \(E_{c}\) to obtain the separated feature representation \(\hat{E}_{q,c}\): \[\hat{E}_{q,c}=\hat{M}_{q,c}\odot E_{c}\] (4) where \(E_{c}\) denotes the encoded feature representation in the \(c^{\text{th}}\) microphone, \(\odot\) denotes the Hadamard product. 3. In the decoder section, the separated feature \(\hat{E}_{q,c}\) passes through a Conv2D layer, inverse DRC (IDRC), and inverse STFT (ISFFT) to obtain the finally separated waveforms \(\hat{y}_{q,c}^{(0)}\), where the superscript \((0)\) denotes the first stage. The whole decoder section can be expressed as: \[\hat{y}_{q,c}^{(0)}=Decoder\{\hat{E}_{q,c}\}_{q}\] (5) where \(\text{Decoder}\{.\}_{q}\) denotes the decoder of the \(q^{\text{th}}\) speaker and \(\hat{y}_{q,c}^{(0)}\) denotes the separated waveform of the \(q^{\text{th}}\) speaker in the \(c^{\text{th}}\) microphone.