  Multi-stage learning is an effective technique to invoke multiple
deep-learning modules sequentially. This paper applies multi-stage learning to
speech enhancement by using a multi-stage structure, where each stage comprises
a self-attention (SA) block followed by stacks of temporal convolutional
network (TCN) blocks with doubling dilation factors. Each stage generates a
prediction that is refined in a subsequent stage. A fusion block is inserted at
the input of later stages to re-inject original information. The resulting
multi-stage speech enhancement system, in short, multi-stage SA-TCN, is
compared with state-of-the-art deep-learning speech enhancement methods using
the LibriSpeech and VCTK data sets. The multi-stage SA-TCN system's
hyper-parameters are fine-tuned, and the impact of the SA block, the fusion
block and the number of stages are determined. The use of a multi-stage SA-TCN
system as a front-end for automatic speech recognition systems is investigated
as well. It is shown that the multi-stage SA-TCN systems perform well relative
to other state-of-the-art systems in terms of speech enhancement and speech
recognition scores.
