# SGM-EMA: Speech Enhancement Method Score-Based Diffusion Model and EMA Mechanism
###### Abstract  
The score-based diffusion model has made significant progress in the field of computer vision, surpassing the performance of generative models, such as variational autoencoders, and has been extended to applications such as speech enhancement and recognition. This paper proposes a U-Net architecture using a score-based diffusion model and an efficient multi-scale attention mechanism (EMA) for the speech enhancement task. The model leverages the symmetric structure of U-Net to extract speech features and captures contextual information and local details across different scales using the EMA mechanism, improving speech quality in noisy environments. We evaluate the method on the VoiceBank-DEMAND (VB-DMD) dataset and the DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus-TUT Sound Events 2017 (TIMIT-TUT) dataset. The experimental results show that the proposed model performed well in terms of speech quality perception (PESQ), extended short-time objective intelligibility (ESTOI), and scale-invariant signal-to-distortion ratio (SI-SDR). Especially when processing out-of-dataset noisy speech, the proposed method achieved excellent speech enhancement results compared to other methods, demonstrating the model's strong generalization capability. We also conducted an ablation study on the SDE solver and the EMA mechanism, and the results show that the reverse diffusion method outperformed the Euler-Maruyama method, and the EMA strategy could improve the model performance. The results demonstrate the effectiveness of these two techniques in our system. Nevertheless, since the model is specifically designed for Gaussian noise, its performance under non-Gaussian or complex noise conditions may be limited. s +
Footnote †: 2025: 15, 5243  
achieve noise reduction by attenuating the noise spectrum. However, real-world noise lacks structured patterns, unlike synthetically generated non-stationary noise, which may still exhibit regular behavior. Another category of methods is based on statistical models, such as hidden Markov models [17] and Gaussian mixture models [18], which rely on specific acoustic assumptions and probabilistic model structures. However, in real scenarios, noise often exhibits variability, making acoustic assumptions about noise difficult and thereby limiting the denoising performance. Unlike traditional methods, deep learning [19; 20; 21; 22] techniques can automatically learn complex patterns from large-scale datasets to achieve better speech enhancement performance. Deep learning models are generally categorized into discriminant models (D) and generative models (G). The core idea of the discriminant model is to learn a mapping from noisy speech to clean speech, typically using time domain or frequency domain methods [23; 24; 25; 26; 27], complex spectrum mapping [28], or directly operating in the time domain [29; 30; 31]. In contrast, generative models learn the underlying statistical properties of clean speech data, enabling them to perform well even when the training and test datasets differ. Typical generative models include approaches such as generative adversarial networks [32; 33; 34] and variational autoencoders [35; 36]. Many areas have been studied regarding diffusion models. [PERSON] et al. [37] proposed a method for restoring degraded speech using an improved diffusion model, modifying the DiffWave architecture to recover the original speech signal better. [PERSON] et al. [38] proposed a generative model that combines score-based diffusion with a multi-resolution conditioning network enhanced by a mixture density network, capable of handling 55 different types of distortion simultaneously. [PERSON] et al. [39] proposed an audiovisual speech enhancement system that leverages a score-based diffusion model informed by visual cues and uses audiovisual embeddings derived from a self-supervised learning model fine-tuned for lip reading. Their system improves speech quality and reduces generation artifacts, such as speech confusion. [PERSON] et al.