# Exploring Multi-Stage GAN with Self-Attention for Speech Enhancement
## 2 Background  
### Conventional Segan  
In a speech enhancement task, a given speech or raw audio signal with noise can be represented as \(\mathbf{\tilde{x}}=\mathbf{x}+\mathbf{n}\in\mathbb{R}^{T}\), where \(\mathbf{x}\in\mathbb{R}^{T}\) is the clean signal and \(\mathbf{n}\in\mathbb{R}^{T}\) represents a noisy signal that corrupts the speech signal. This noisy signal is often considered additive background noise. The main goal of speech enhancement is to remove the noisy signal in the raw audio signal by finding an enhancement mapping function \(f\) such that \(f(\mathbf{\tilde{x}}):\mathbf{\tilde{x}}\rightarrow\mathbf{x}\). One such mapping function that has been adopted with great success is the conditional-based GAN method [39; 40].  
The conditional-based GAN method was first employed in speech enhancement in the seminal work by [PERSON] et al. [16], which is referred to as speech Enhancement GAN (SEGAN). In SEGAN, the generator \(G\) (for emphasis, a single generator) is provided with a corrupted raw audio signal as the enhancement mapping function such that \(\mathbf{\hat{x}}=G(\mathbf{z},\mathbf{\tilde{x}})\), where \(\mathbf{z}\) is a latent variable. Then, the generator \(G\) is trained to produce the enhanced output signal \(\mathbf{\hat{x}}\) simultaneously with the discriminator \(D\), which learns to distinguish between the enhanced output signal \(\mathbf{\hat{x}}\) and the real clean signal \(\mathbf{x}\) by classifying the pair of signals \((\mathbf{x},\mathbf{\tilde{x}})\) as real and \((\mathbf{\hat{x}},\mathbf{\tilde{x}})\) as fake pair. The training procedure is illustrated in Figure 2, and the Equations (1) and (2) are the least-squares objective functions for training the discriminator \(D\) and the generator \(G\), respectively.  
\[\begin{split}\underset{D}{}^{min}\mathcal{L}_{LS}(D)& =\frac{1}{2}\mathbb{E}_{\mathbf{x},\mathbf{\tilde{x}}\sim p_{data}( \mathbf{x},\mathbf{\tilde{x}})}(D(\mathbf{x},\mathbf{\tilde{x}})-1)^{2}+ \frac{1}{2}\mathbb{E}_{\mathbf{z}\sim p_{\mathbf{z}}(\mathbf{z}),\mathbf{ \tilde{x}}_{data}(\mathbf{\tilde{x}})}D(G(\mathbf{z},\mathbf{\tilde{x}}), \mathbf{\tilde{x}})^{2},\\ \underset{G}{}^{min}\mathcal{L}_{LS}(G)&=\frac{1}{ 2}\mathbb{E}_{\mathbf{z}\sim p_{\mathbf{z}}(\mathbf{z}),\mathbf{\tilde{x}}_{ data}(\mathbf{\tilde{x}})}(D(G(\mathbf{z},\mathbf{\tilde{x}}),\mathbf{ \tilde{x}})-1)^{2}+\lambda||G(\mathbf{z},\mathbf{\tilde{x}})-\mathbf{\tilde{x }})||_{1}.\end{split} \tag{1}\]  
Several improvements have been introduced to the SEGAN algorithm focusing on improving the quality of the enhanced speech signals [17; 23; 41; 42; 43]. For instance, [PERSON] and [PERSON] [17] presented an improvement in enhancement by introducing a cost function with a gradient penalty. The self-attention module [37] was incorporated into the SEGAN to prevent the convolutional layers from obscuring the temporal dependency in an input sequence of signals [23]. Additionally, MetricGAN [24] also demonstrated that evaluation metrics can be used to improve the performance of conditional GAN-based method in speech enhancement by using the metrics that are directly related to speech signals (e.g., PESQ and STOI) rather than Euclidean distances to compute the loss in the SEGAN.