## 2 Proposed Speech Enhancement Methods for LDVs
### Waveform-Based Speech Enhancement  
Figure 6 shows the processing procedure of the waveform-based method. As described above, speech shows the obvious harmonic structure in the low-frequency bands and white noise distribution in the high-frequency bands. Therefore, with the waveform-based method, low-frequency and high-frequency components are processed separately. Figure 4: Structure of DNN for amplitude spectrum reconstruction. Figure 5: Structure of DNN for phase spectrum reconstruction. Figure 3: Block diagram of proposed STFT-based method. In the learning stage, since the high-frequency components of the observed speech \(\mathbf{x}\) do not have the harmonic structure of speech, the high-frequency components of \(\mathbf{x}\) and clean speech \(\mathbf{y}\) are first removed by down-sampling to increase the accuracy of low-frequency component restoration. Down-sampled speech \(\mathbf{x}^{\text{NB}}\) and \(\mathbf{y}^{\text{NB}}\) are then input to the noise suppression DNN to train the network. Then, the low-frequency-enhanced speech \(\hat{\mathbf{x}}^{\text{NB}}\) are up-sampled to match the length of the clean speech. Finally, the up-sampled speech \(\hat{\mathbf{x}}^{\text{WB}}\) and \(\mathbf{y}\) are each 8-bit quantized by \(\mu\)-law [20] and used to train the DNN for high-frequency restoration. The MSE is the most common loss function for deep learning. If set MSE as the loss function, its gradient is given by the derivative of MSE that \(\hat{\mathbf{x}}^{\text{NB}}-\mathbf{y}^{\text{NB}}\). Here, the power of the speech signal concentrates in the low-frequency range. In other words, \(\hat{\mathbf{x}}^{\text{NB}}-\mathbf{y}^{\text{NB}}\) is almost determined by the amplitude difference in the low-frequency bands, which is beneficial to the restoration of low-frequency signals. Moreover, when training a network on the basis of the MSE, the waveform of the recovered high-frequency component may be smoothed more than clean speech [21]. Therefore, the network with the MSE as the loss function is used for low-frequency band noise suppression. As shown in Figure 7, the noise-suppression DNN consists of convolutional layers with a dilated convolution structure of eight layers. The loss function and activation functions use the MSE and parametric ReLu (PReLU) [22], respectively. In the high-frequency component restoration process, recurrent neural networks (RNN) are used to predict the sample at time \(n\) by using the previous samples. By 8-bit quantization of the waveform, the amplitude of the waveform becomes an integer of 0-255, which will be input to the network. The output of the last fully connected layer is the probability distribution of each sample, which is a 256-dimensional vector. Figure 8 shows the network structure. The DNN for high-frequency component reconstruction consists of  
Figure 6: Block diagram of the proposed waveform-based method. Figure 7: Structure of noise suppression DNN. two LSTM layers and two fully-connected layers and processes the input waveform sample by sample. Equation (2) shows the processing of the LSTM layer at time \(n\). \[S(n)=\mathcal{G}(S(n-1),\hat{x}^{\text{WB}}(n)) \tag{2}\]  
Here, \(S(n)\) is the LSTM output, \(\hat{x}^{\text{NB}}(n)\) is the network input, and \(\mathcal{G}(\cdot)\) is the activation function of the LSTM. The output \(\hat{y}^{\text{WB}}(n)\) at time step \(n\) is based on the conditional probability of Equation (3). \[p(g^{\text{WB}}(n)|\hat{x}^{\text{WB}}(0),\hat{x}^{\text{WB}}(1),\hat{x}^{ \text{WB}}(2),...,\hat{x}^{\text{WB}}(n))=\text{FC}(S(n)) \tag{3}\]  
Here, \(\text{FC}(\cdot)\) is the output of the fully-connected layer. The DNN for high-frequency restore is a classification network with cross-entropy as the loss function. In the speech enhancement stage, high-frequency components are first removed from the observed speech \(x\) by down-sampling. The down-sampled speech \(x^{\text{NB}}\) is then subjected to noise suppression by using the noise suppression DNN to enhance the low-frequency components.