# TASE: Task-Aware Speech Enhancement for Wake-Up Word Detection in Voice Assistants +
## 2 Task-Aware Speech Enhancement
### Model  
Our model is a fully-convolutional denoising autoencoder with skip connections (Figure 1), in the style of previous effective SE models [27; 30; 34]. In training, we input a noisy audio waveform \(\mathbf{x}\in\mathbb{R}^{T}\), comprised of clean speech signal \(\mathbf{y}\in\mathbb{R}^{T}\) and background noise \(\mathbf{n}\in\mathbb{R}^{T}\) so that \(\mathbf{x}=\lambda\mathbf{y}+(1-\lambda)\mathbf{n}\), where \(\lambda\) is a parameter to control the SNR. The encoder consists of six convolutional blocks (ConvBlock1D), each being a sequence of a convolutional layer, an instance normalization and a rectified linear unit (ReLU). Kernel size \(K=4\) and stride \(S=2\) are used, except in the first layer, where \(K=7\) and \(S=1\). After the encoder, the compressed signal goes through three intermediate residual blocks (ResBlock1D) that preserve the shape, each formed by two ConvBlock1D with \(K=3\) and \(S=1\). Skip connections are added from the input of each residual block to its output. The signal finally flows through the decoder, which follows the inverse structure of the encoder, where deconvolutional blocks (DeconvBlock1D) replace the convolutional layers of the ConvBlock1D with transposed convolutional layers. The output of the decoder is the enhanced signal with the shape of the input waveform, which is ready to be passed on to the WUW classifier. Both the encoder and decoder blocks are connected with skip connections to ensure that low-level details of the waveform are preserved. The model is fully convolutional because this reduces the forward delay compared with the same architecture using a Recurrent Neural Network (RNN) to the latent representation of the audio. Table 1 presents a comparison between state-of-the-art architectures and ours, in terms of number of parameters, operations, size, and forward delay. The measurements of the forward time have been done in the same conditions: same CPU and using the same input data, an audio of 1.5 s. We performed 100 forwards and then calculated the average forward time. For the architecture named \"gruse\", we replaced the residual blocks that process the compressed signal of our architecture by a Gated Recurrent Unit (GRU) with a hidden size of 256. This produces a smaller model that carries out less operations, while considerably increasing the forward delay. Architectures demucs (H = 64 and H = 48) are from the work proposed in Reference [30], and NSNet2 is the baseline network used for the Deep Noise Suppression Challenge [35]. Optimization is guided with a regression loss function (L1 loss) at raw waveform level, together with another L1 loss over the log-mel spectrogram, as proposed in Reference [36], to reconstruct the clean signal \(\hat{\mathbf{y}}\) at the output. Finally, we include the binary cross-entropy classification loss (BCE loss) of the WUW classifier in the TASE use-case. We either train the TASE model jointly with the WUW classifier from scratch, or we just concatenate a frozen pre-trained classifier at its output. In any case, the BCE loss is available to TASE to optimize itself toward WUW detection. Our final loss function is defined as a linear combination of the three losses:  
\[L_{T}=\alpha L_{raw}(\mathbf{y},\hat{\mathbf{y}})+\beta L_{spec}(S(\mathbf{y}),S(\hat{\mathbf{y }}))+\gamma L_{BCE}, \tag{1}\] \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Architecture** & **Parameters** & **\# Operations** & **Size (MB)** & **Forward Time (ms)** \\ \hline demucs (H = 64) & 33.53M & 10,014.87M & 278.61 & 163.21 \\ demucs (H = 48) & 18.86M & 5644.95M & 184.05 & 98.28 \\ NSNet2 & 2.80M & - & - & 22.00 \\ TASE & 2.45M & 4156.26M & 154.67 & 65.50 \\ gruse & 1.31M & 1852.57M & 64.11 & 176.42 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters, number of operations (multiplications and additions), size, and forward time of SE models.