  Many purely neural network based speech separation approaches have been
proposed that greatly improve objective assessment scores, but they often
introduce nonlinear distortions that are harmful to automatic speech
recognition (ASR). Minimum variance distortionless response (MVDR) filters
strive to remove nonlinear distortions, however, these approaches either are
not optimal for removing residual (linear) noise, or they are unstable when
used jointly with neural networks. In this study, we propose a multi-channel
multi-frame (MCMF) all deep learning (ADL)-MVDR approach for target speech
separation, which extends our preliminary multi-channel ADL-MVDR approach. The
MCMF ADL-MVDR handles different numbers of microphone channels in one
framework, where it addresses linear and nonlinear distortions. Spatio-temporal
cross correlations are also fully utilized in the proposed approach. The
proposed system is evaluated using a Mandarin audio-visual corpora and is
compared with several state-of-the-art approaches. Experimental results
demonstrate the superiority of our proposed framework under different scenarios
and across several objective evaluation metrics, including ASR performance.
