  Data-driven speech enhancement employing deep neural networks (DNNs) can
provide state-of-the-art performance even in the presence of non-stationary
noise. During the training process, most of the speech enhancement neural
networks are trained in a fully supervised way with losses requiring noisy
speech to be synthesized by clean speech and additive noise. However, in a real
implementation, only the noisy speech mixture is available, which leads to the
question, how such data could be advantageously employed in training. In this
work, we propose an end-to-end non-intrusive PESQNet DNN which estimates
perceptual evaluation of speech quality (PESQ) scores, allowing a
reference-free loss for real data. As a further novelty, we combine the PESQNet
loss with denoising and dereverberation loss terms, and train a complex
mask-based fully convolutional recurrent neural network (FCRN) in a "weakly"
supervised way, each training cycle employing some synthetic data, some real
data, and again synthetic data to keep the PESQNet up-to-date. In a subjective
listening test, our proposed framework outperforms the Interspeech 2021 Deep
Noise Suppression (DNS) Challenge baseline overall by 0.09 MOS points and in
particular by 0.45 background noise MOS points.
