Phase-Aware Speech Enhancement and Dereverberation
Creators: name: Zhang, Jingshu
Publisher: name: University of Surrey
Year: 2023
Subjects: Speech enhancement
Speech dereverberation
Phase processing
Deep learning

description: This thesis aims to investigate the phase estimation problem in speech enhancement and dereverberation.
description: Noise and reverberation widely exist in daily lives, and have detrimental effects on the quality and
description: intelligibility of speech for human listeners. Therefore, speech enhancement and dereverberation
description: are very critical when processing speech with noise and reverberation. In many approaches of
description: speech enhancement and dereverberation, only the magnitude is considered
description: for enhancement, while the noisy phase is untouched and used directly when reconstructing
description: speech from enhanced magnitude. However, it has been discussed that phase is important in
description: speech enhancement, and using noisy phase to reconstruct speech may degrade its quality and
description: intelligibility. Therefore, we focused on phase-aware methods in this Thesis.<br>
description: <br>
description: In our initial speech dereverberation experiments, we found an issue in the perceptual evaluation
description: of speech quality (PESQ) measure. When measuring reverberant speech, the cross-correlation-based
description: time alignment process in the PESQ measure may incorrectly align the reverberant
description: speech with the reference speech, especially when the reverberation is heavy. In the time
description: alignment process, the speech would be split into utterances based on speech activities, and
description: these utterances would be further split into small clips. We proposed a modified PESQ called
description: time alignment restricted PESQ (TAR-PESQ) where this utterance splitting step is removed.
description: We conducted experiments using both simulated room impulse responses and real RIRs to test
description: the time alignment performance of the proposed TAR-PESQ and the original PESQ. We also
description: computed the variance of PESQ scores estimated using both TAR-PESQ and PESQ to measure
description: the consistency of the produced PESQ scores. The results showed that our proposed TAR-PESQ
description: correctly aligned more pieces of speech than the original PESQ, and the TAR-PESQ could
description: produce more consistent PESQ scores than the original PESQ.<br>
description: <br>
description: When estimating complex speech spectrograms or masks in the complex domain, the mean
description: square error (MSE) loss function is commonly used in the training. However, the MSE loss
description: function sometimes puts more weight on phase than magnitude, which may degrade the performance
description: of the trained model. To balance the estimation of magnitude and phase, we proposed a
description: weighted magnitude-phase loss function for speech dereverberation. We conducted experiments
description: and objective measures showed that the weighted magnitude-phase loss function improved the
description: performance in both speech denoising and dereverberation when a suitable weight is chosen.
description: We also conducted a listening test to measure the perceptual quality of the produced speech, and
description: the results showed that our proposed WMP loss function is slightly preferred in terms of speech
description: dereverberation.<br>
description: <br>
description: To better exploit the temporal and frequency structure of phase, we proposed to use DNN
description: models to estimate phase derivative features as well as phase directly from the magnitude and
description: phase of noisy input. Assuming the phase time-frequency (T-F) units with low energy have
description: little impact on the quality of speech, we proposed a masked loss function to mask out the
description: time-frequency (T-F) units with low energy when computing loss. The experiments showed that
description: this loss function led to an increase in speech quality. We then proposed a non-iterative phase
description: reconstruction approach and an iterative phase reconstruction approach to reconstruct the phase
description: from estimated phase features. The experiments showed that these reconstruction approaches
description: achieved results similar to the estimated phase in terms of the quality of reconstructed speech.<br>