# Lightweight End-to-End Speech Enhancement Generative Adversarial Network Using Sinc Convolutions
## 5 Sinc-SEGAN Architecture  
We investigate two different deployments of the Sinc convolution illustrated in Section 4: (i) sitting before the first layer of the generator's encoder and the discriminator, and behind the last layer of the generator's decoder, referred to as the addition architecture, and (ii) acting as a substitute for the first standard convolutional layer of the generator's encoder and the discriminator, and the last standard convolutional layer of the generator's decoder, referred to as the substitution architecture. In the case of the addition architecture, the generator makes use of an encoder-decoder architecture. The first layer of the encoder is Sinc convolution, using 64 filters of length 251 and stride = 1, followed by a max pooling layer (stride = 2). Thereafter, there are five one-dimensional strided convolutional layers with a common filter width of 31 and stride = 4, each followed by parametric rectified linear units (PReLUs) [36]. At the 6 th layer of the encoder, the encoding vector \(\mathbf{c}\in\mathbb{R}^{8\times 1024}\) is stacked with the noise sample \(\mathbf{z}\in\mathbb{R}^{8\times 1024}\), sampled from the distribution \(\mathcal{N}(0,I)\), and presented to the decoder. The decoder component mirrors the encoder architecture, with the same number of filters and filter width, to reverse the encoding process through deconvolutions, namely fractional-strided transposed convolution. The last layer of the decoder is also a Sinc convolution (filter number = 64, kernel size = 251, and stride = 1), and there is an un-maxpooling layer (stride = 2) before it for upsampling. Learnable skip connections are deployed to connect the encoding layer with its corresponding decoding layer to allow the information to be summed to the decoder feature maps. The learnable vectors \(\mathbf{a}_{l}\) multiply every channel of its corresponding shuttle layer \(l\) by a scalar factor \(\gamma_{l,k}\). Hence, for the input of the \(j\)th decoder layer \(\mathbf{h}_{j}\), we have the addition of the corresponding \(l\)th encoder layer response, following  
\[\mathbf{h}_{j}=\mathbf{h}_{j-1}+\mathbf{a}_{l}\odot\mathbf{h}_{l}, \tag{9}\]  
where \(\odot\) is an element-wise product along channels. The discriminator is constructed of a similar architecture to the encoder of the generator. However, it receives a two-channel input, i.e., \((\mathbf{x}^{*},\mathbf{\tilde{x}})\) or \((\mathbf{\tilde{x}},\mathbf{\tilde{x}})\), and utilizes virtual batch-norm [37] before LeakyReLU [38] activation with \(\alpha=0.3\). Moreover, the discriminator is topped up with a \(1\times 1\) convolutional layer to reduce the dimension of the output of the last convolutional layer for the subsequent classification task by the softmax layer. To sum up, in the addition architecture, the generator consists of 12 layers and the discriminator consists of six layers. We illustrate the addition architecture of Sinc-SEGAN in Figure 1.  
The substitution architecture, as illustrated in Figure 2, is similar to the first case, but the original first layer of the encoder is substituted as Sinc convolution, using 64 filters of length 251. Its stride = 4, in line with the standard convolutional layer, and the max pooling layer is removed. Thereafter, there are four one-dimensional strided convolutional layers with a common filter width of 31 and stride = 4, each followed by PReLUs [36]. At the 6 th layer of the encoder, the encoding vector \(\mathbf{c}\in\mathbb{R}^{16\times 1024}\) is stacked with the noise sample \(\mathbf{z}\in\mathbb{R}^{16\times 1024}\), sampled from the distribution \(\mathcal{N}(0,I)\), and presented to the decoder. The decoder component still uses the mirror architecture to reverse the encoding process through deconvolutions. The last deconvolutional layer of the decoder is also replaced with a Sinc convolution (filter number = 64, kernel size = 251, and stride = 4), and the upsampling layer is not needed anymore.