# Lightweight End-to-End Speech Enhancement Generative Adversarial Network Using Sinc Convolutions
## 1 Introduction  
Speech enhancement is the task of removing or attenuating background noise from a speech signal, and it is generally concerned with improving the intelligibility and quality of degraded speech [1]. Speech enhancement is widely used as a preprocessor in speech-related applications including robust automatic speech recognition systems [2] and communication systems, e.g., speech coding [3], hearing aid design [4], and cochlear implants [5]. Conventional speech enhancement approaches include the Wiener filter [6], time-frequency masking [7, 8, 9], signal approximation [10, 11], the spectral mapping [12, 13], etc. In the last few years, supervised methods for speech enhancement which leverage deep learning methods have become mainstream. Well-known models include deep de-noising autoencoders [14], convolutional neural networks (CNNs) [15], and recurrent neural networks [16, 17]. There exists a class of generative methods relying on generative adversarial networks (GANs) [18], which have CNNs as the backbone and have been verified to be efficient for speech enhancement [2, 19, 20, 21, 22, 23]. GANs designate a generator for enhancement mappingand a discriminator for distinguishing real signals from fake ones. With the transmitted information from the discriminator, the generator learns to produce outputs that resemble the realistic distribution of clean signals. Most past attempts deal with magnitude spectrum inputs, as it is often claimed that short-time phase is unimportant for speech enhancement [24]. However, further studies [25] demonstrate that a clean phase spectrum can deliver significant improvements to speech quality. CNNs are the most popular deep learning architecture for processing raw speech inputs. This is due their weight sharing, local filters, and pooling features, which allow extraction of robust and invariant representations. The first convolutional layer is a critical part of waveform-based CNNs [26], since it processes high-dimensional inputs and extracts low-level speech representations for deeper layers. However, it is susceptible to vanishing gradient problems. To alleviate this issue, [PERSON] et al. [26] proposed the Sinc convolution to learn more meaningful filters in the input layer. Differently from a standard CNN, the Sinc-convolution layer convolves the waveform with a set of parametrized Sinc functions that implement band-pass filters, and only needs to learn the low and high cutoff frequencies. Consequently, the Sinc convolution is faster-converging and lightweight. It performs extraordinarily well in capturing selective speech clues, e.g., the pitch region, the first formant, and the second formant, which are essential for resembling clean speech signals. It has also achieved remarkable success in some fields of speech signal processing, e.g., speech recognition [27; 28], speaker identification [26], keyword spotting [29], etc. Unfortunately, Sinc convolution for speech enhancement is still an under-explored research direction. There is no available model fusing these two powerful paradigms--the Sinc convolution operating over raw speech waveforms and the generative adversarial models for speech enhancement. Therefore, we propose to bridge this gap by usefully merging the Sinc convolution and the speech enhancement GANs (SEGAN), resulting in a customizable, lightweight, and interpretable system, termed Sinc-SEGAN. Contributions of this paper are summarized as:  
* Transfer the success achieved by the Sinc convolution in the field of speech and speaker recognition to the field of end-to-end speech enhancement. * Optimize the SEGAN architecture from the seminal work [19], and enhance the original Sinc convolution layer to fit the advanced SEGAN. * Analyze the learned filters of the Sinc convolution layer. * Apply data augmentation techniques on raw speech waveforms directly to further improve the system performance.