## 2 Baseline UDS-Based SSI  
### Estimating the Speech Spectrum  
When the ultrasonic tone with a frequency \(f_{c}\) is reflected on an articulating face, Doppler frequency shifts that are potentially caused by the movements of articulatory organs appear in the reflected signals. Assuming that \(M\) objects are engaged with the Doppler frequency shift, the instantaneous velocity of the \(m\)-th object at time \(t\) is then given by \(v_{m}(t)\), and the reflected signal is given by  
\[R(t) = \sum_{m=1}^{M}A_{T}k_{m}\cos(\phi_{m}+\Psi_{T}),\] \[\phi_{m} = 2\pi f_{c}\left[t+\frac{2}{v_{s}}\int_{0}^{t}v_{m}(\tau)d\tau \right]+\Psi_{m} \tag{1}\]  
where \(k_{m}\) and \(\Psi_{m}\) are the attenuation coefficient and phase shift of the \(m\)-th object at frequency \(f_{c}\), respectively. \(A_{T}\) and \(\Psi_{T}\) are the amplitude and phase of the transmitted sinusoid, and \(v_{s}\) is the speed of sound. The principle of the ultrasound-based speech estimator is that the variables (\(k_{m}\) and \(\Psi_{m}\)) associated with the Doppler frequency shifts are highly correlated with the underlying speech signals. A block diagram of a baseline UDS-based SSI with deep neural networks (DNN) is shown in Figure 1 where the training procedure and the synthesis procedure are presented at the top and the bottom, respectively. The first step of UDS-based speech estimation is to extract the features from the received ultrasonic signal (US). Our previous study [18] showed that compared with other parameters, the mel-frequency filter bank energies of a demodulated US signal yielded a superior performance in prediction of speech parameters. The bandwidth of the demodulated ultrasonic signal was 2 kHz, and was determined according to the maximum frequency of articulatory movements. The resultant number of mel-bands was 16.  
A supervised learning framework was adopted to estimate the relationship between the input UDS and the output speech. In the training stage, a regression DNN model was trained from a training corpus, that consisted of pairs of audio and ultrasound signals. Speech represented by the log magnitude spectra was the target output of the DNN. In the estimation stage, the feature parameters derived from the ultrasound signals were inputted to the trained DNN. The short-time estimated speech signal was obtained by inverse Fourier transform using the output of the DNN, which was the estimated speech spectrum, and the random phase spectrum. Finally, continuous waveforms were obtained by concatenating the short-time estimated speech signals. using the OverLap and Addition (OLA) method.  
A back-propagation algorithm is typically used to train a DNN. The objective function is given by a mean square error between the estimated log magnitude spectrum and that of original speech, as follows:  
\[E=\frac{1}{N}\sum_{n=1}^{N}\sum_{p=1}^{P}\Big{\{}\mathcal{F}_{p}(\mathbf{W}, \mathbf{x}_{n})-Y_{p,n}\Big{\}}^{2} \tag{2}\]  
where \(\mathcal{F}_{p}(\mathbf{W},\mathbf{x})\) is the \(p\)-th output of the DNN with the weights \(\mathbf{W}\) where the input UDS features \(\mathbf{x}\) are given. \(Y_{p,n}\) denotes the \(p\)-th frequency bin of the log-spectral feature at frame index \(n\). A stochastic gradient descent algorithm was performed in mini-batches with multiple epochs to improve the learning convergence. An updated estimate of the weights \(\mathbf{W}\) with a learning rate \(\lambda\) was computed iteratively as follows:  
\[\mathbf{W}_{n+1}=\mathbf{W}_{n}-\lambda\bigtriangledown\mathbf{W}\,E \tag{3}\]  
The DNN captures the acoustic context information along the time axis by adopting multiple frames of ultrasonic signals over time as DNN input [39]. In the present study, the number of neighboring frames was heuristically determined to be \(5\) by maximizing the performance in terms of speech estimation.