# Exploring Multi-Stage GAN with Self-Attention for Speech Enhancement
## 6 Conclusions  
This paper introduced a self-attention block into multi-generator speech enhancement GANs (ISEGAN and DSEGAN) to improve their temporal dependency capability for enhancing speech signals. The self-attention block can be integrated at different convolutional layers of the multiple generators of the ISEGAN and the DSEGAN networks or given sufficient memory, it can be integrated at all the convolutional layers of the SEGAN's and the DSEGAN's multiple generators. Our experiments demonstrate that integrating a self-attention mechanism into ISEGAN and DSEGAN networks, which we respectively called ISEGAN-Self-Attention and DSEGAN-Self-Attention networks, can significantly improve the performance of the speech enhancement system. The experimental results show that the ISEGAN-Self-Attention and the DSEGAN-Self-Attention significantly improve the enhancement performance of ISEGAN and DSEGAN. In addition, the ISEGAN-Self-Attention and the DSEGAN-Self-Attention outperformed the SEGAN and the SASEGAN baselines in all the objective evaluation metrics. Furthermore, we observe that the use of self-attention helps bridge the performance gap between the ISEGAN and the DSEGAN, demonstrating the effectiveness of the self-attention mechanism for both the shared-parameter multi-generator speech enhancement GAN (ISEGAN) and the non-shared-parameter multi-generator speech enhancement GAN (DSEGAN). Overall, our findings highlight the potential benefits of self-attention as a valuable technique for enhancing the performance of multi-generator speech enhancement systems.  
Conceptualization, B.K.A.A. and C.B.-B.; methodology, B.K.A.A.; investigation, B.K.A.A.; writing--original draft preparation, B.K.A.A. and C.B.-B.; writing--review and editing, H.I. and C.B.-B.; supervision, H.I. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
The work of the first author was supported by the Makiguchi Foundation under the Makiguchi Scholarship for International Students.  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Metric**} & \multirow{2}{*}{**DSEGAN**} & \multirow{2}{*}{**ISEGAN**} & \multicolumn{2}{c}{**ISEGAN-Self-Attention**} & \multicolumn{2}{c}{**DSEGAN-Self-Attention**} \\ \cline{4-7}  & & & \(G_{1}\) & \(G_{2}\) & \(G_{1}\) & \(G_{2}\) \\ \hline PESQ & 2.71 & 2.66 & **2.63** & 2.57 & **2.68** & 2.64 \\ CSIG & 3.58 & 3.58 & **3.51** & 3.49 & **3.52** & 3.50 \\ CBAK & 3.15 & 3.15 & **3.09** & 3.08 & **3.11** & 3.08 \\ COVID & 3.11 & 3.04 & **3.07** & 3.04 & **3.09** & 3.05 \\ SSNR & 9.19 & 9.04 & **9.11** & 9.03 & **9.09** & 9.02 \\ STOI & 93.29 & 93.25 & **93.38** & 93.32 & **93.42** & 93.36 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The results of the ablation study on \(N=2\) (i.e., 2 generators) setup of the ISEGAN-Self-Attention and the DSEGAN-Self-Attention networks. The results are compared with ISEGAN and DSEGAN. The **boldface** is the setup with the highest score per an objective evaluation metric.