# Iteratively Refined Multi-Channel Speech Separation
## 1 Introduction  
Currently, speech separation technology is playing a crucial role in human-computer interaction, audio processing, and communication systems [1, 2]. With the advancement of technology, especially in deep learning, significant progress has been made in achieving efficient speech separation, especially in the fields of single-channel based speech separation [3, 4, 5, 6]. Although single-channel based methods perform well in some environments, their effectiveness is limited in cases of more complex acoustic environments. Therefore, in order to overcome these limitations and further improve the performance of speech separation, multi-channel speech separation methods [7] have been explored. Traditional multi-channel speech separation methods, such as delay-and-sum beamforming [8], work well in certain situations. However, in reverberant environments, with the increase of sound sources, it is often difficult for the traditional methods to effectively separate speech signals. The difficulty is mainly focused on the fact that traditional beamforming techniques rely on relatively simple signal processing strategies, which are not ideal for dealing with dynamic changes of sound source positions or for dealing with complex and variable acoustic environments. Due to the limitations of conventional multi-channel methods, particular attention has been paid to approaches based on neural beamforming, combining the powerful nonlinear modeling capabilities of neural networks with the action of a beamformer. In classical neural beamforming, the neural network is used to obtain initial speech separation. Subsequently, this initially separated speech and the original speech are together used in the beamformer to compute spatial covariance matrices (SCMs) [9]. In addition, the post filter cascaded to the beamformer is incorporated to further optimize the quality and intelligibility of the separated speech [10]. Compared with conventional beamforming methods, neural beamforming has been demonstrated to have significant advantages in dealing with complex acoustic environments, which has made it a mainstream method in research and application in recent years [9; 11; 12; 13]. For example, a masking-based neural beamforming method was developed in [11] and [13], in which multiple single-channel long short-term memory (LSTM) networks were first used to estimate the masks of the speakers, and these masks were then used to estimate the SCM of speech and noise for the MVDR beamformer. This kind of method has shown a significant improvement in the performance of speech separation compared with conventional beamforming methods. In addition, a signal-based neural beamforming method was proposed [9], in which a time-domain audio separation network (TasNet) was used to pre-separate the speech, and the separated speech was used to calculate the SCM used in the MVDR beamformer. This method achieved better performance than using an ideal ratio mask in the MVDR beamformer [9]. Research work [14] indicated that reverberation had a significant impact on the separation while using the TasNet. This observation inspired us to explore a speech separation method with anti-reverberation ability, aiming at achieving a more accurate SCM for use in the MVDR beamformer for improving beamforming performance. Consequently, in our previous work [15], a time-frequency domain dual-path recurrent neural network (TFDPRNN) was proposed for achieving better performance of speech separation in a reverberant environment. A significant performance improvement was achieved by combining the MVDR beamformer and TFDPRNN (called Beam-TFDPRNN). Although these neural beamforming methods have a good performance, they are still restricted to the linear filtering operation and their performance is limited. Therefore, in this paper, we explore other ways to improve the performance of neural beamforming. In recent years, a notable trend in speech separation has focused on iteratively refined structures.