## 2 U-Net-Based Speech Enhancement  
The U-Net-based SE method is inspired by the observation that the U-Net was developed for medical image segmentation [25], and regions of interest in medical images could be used as target speech spectrograms in noise. Thus, the U-Net can be constructed by using the spectrogram of noisy input speech as an input feature to estimate the IRM between a pair of noisy input and clean target spectrograms. Figure 1 shows the network architecture of the U-Net-based SE method [15], which consists of an encoding network and a decoding network with skip connections between the two networks. As the input features, the noisy input speech sampled at 16 kHz is segmented into consecutive frames of 25 ms with a 10 ms overlap. Then, a 512-point fast Fourier transform (FFT) was applied to each frame, and 256 spectral magnitudes from 32 frames are concatenated into a (\(256\times 32\times 1\))-dimensional image. Next, the input image is passed into the encoding network composed of three 2-dimensional (2D) convolution layers at a stride of \(2\times 2\). Each convolution layer with a kernel size of \(5\times 5\) is followed by a batch normalization (BN) layer, where the leaky rectified linear unit (ReLU) activation function is applied. The decoding network is also composed of three 2D deconvolution layers at a stride of \(2\times 2\), which operate in a reverse sequence to that of the encoding network. The outputs of each deconvolution layer are concatenated with those of their corresponding convolution layer, and they are brought together as the input features for the next deconvolution layer. After that, the outputs of the last deconvolution layer are passed through a sigmoid activation function to estimate the IRM between the 2D spectral images between the noisy input speech and the target speech. Finally, the estimated IRM is multiplied to the noisy input speech spectrum, which results in the estimation of the clean speech spectrum; then, the estimated clean speech in the time domain is reconstructed by applying an inverse FFT (IFFT) to the estimated clean speech spectrum. Figure 1: Network architecture of the U-Net-based single-channel speech enhancement. To train the U-Net, Xavier initialization [26] is utilized for the initial weights of the configured model, and the biases are initialized to zero. The objective of U-Net-based SE is to minimize the mean square error (MSE) between the spectrogram of the target clean speech, \(\left|X_{i}(k)\right|\), and that of the estimated clean speech. That is, the spectrogram of the estimated clean speech, \(\left|\hat{X}_{i}(k)\right|\), is obtained by multiplying the estimated IRM, \(\hat{H}_{i}(k)\left(0\leq\hat{H}_{i}(k)\leq 1\right)\), with the spectrogram of noisy input speech, \(\left|Y_{i}(k)\right|\). Thus, the loss function for the U-Net is defined by  
\[L_{U-Net}(X)=\frac{1}{NK}\sum_{i=0}^{N-1}\sum_{k=0}^{K-1}\bigl{(}\hat{H}_{i}(k )\bigl{|}Y_{i}(k)\bigr{|}-\bigl{|}X_{i}(k)\bigr{|}\bigr{)}^{2} \tag{1}\]  
where \(k\) and \(i\) denote the frequency bin and frame, respectively. In addition, \(N\) is the total number of frames in each minibatch (\(N=32\) in this paper), and \(K\) is the half number of FFT points (\(K=256\)). Adaptive moment estimation (Adam) optimization [27] is utilized for the backpropagation algorithm, and the first- and second-moment decay rates are set as 0.9 and 0.999, respectively, with a learning rate of 0.001. Training is done up to 50 epochs. In addition, the dropout technique is utilized with a keep probability of 0.9 [28]. Note here that the U-Net estimates \(\hat{H}_{i}(k)\) instead of directly estimating \(\left|\hat{X}_{i}(k)\right|\). In other words, the estimated clean speech spectrum is obtained by multiplying \(\hat{H}_{i}(k)\) with \(Y_{i}(k)\). Then, the clean speech signal is estimated by applying an IFFT to \(\hat{H}_{i}(k)Y_{i}(k)\), such as:  
\[\hat{x}_{i}(n)=\text{IFFT}\bigl{\{}\hat{H}_{i}(k)Y_{i}(k)\bigr{\}}=\text{IFFT} \bigl{\{}\hat{H}_{i}(k)\bigl{|}Y_{i}(k)\bigr{|}\exp(j\angle Y_{i}(k))\bigr{\}}. \tag{2}\]