# Speech Inpainting Based on Multi-Layer Long Short-Term Memory Networks
## 4 Experiments
### Model and Training Setup  
In this research, we built eight datasets and introduced 24 LSTM models, comprising 20 single-speaker and 4 multi-speaker models. Specifically, we trained five models for each of the four single-speaker datasets, with the models corresponding to two to six LSTM layers, respectively. We also trained a model with five LSTM layers for each of the four multi-speaker datasets.  
Following dataset construction, we obtained each of the single-speaker datasets with one dimension and 6,400,000 speech samples, and the multi-speaker dataset with ten dimensions and 6,400,000 speech samples for each speaker. For the single-speaker datasets, the models were designed to learn the speech features of different single speakers in the time domain and restore the lost speech samples of the gaps. For the multi-speaker datasets, ten dimensions of training speech samples were input into the model in parallel, and the models were designed to learn the speech features of ten different speakers and restore the lost speech samples of the gaps. All the datasets were divided into training and testing sets, with a split ratio of 85% for training and 15% for testing.  
Speech signals are inherently non-stationary, deviating from the normal distribution due to the dynamic nature of the physical motion process within the vocal organs. To mitigate the impact of dimensionality on the inpainting results and ensure comparability across different speech features, the speech samples were pre-processed using min-max normalisation [47] to scale the values between 0 and 1 for subsequent training.  
The values of the hyperparameters are shown in Table 5. We defined the length of each window as 640 speech samples (sequence length), and the length of each predicted block was set to 80 speech samples (predicted sequence length), which means that the speech inpainting process utilised the preceding 640 speech samples to predict the subsequent 80 speech samples. In the training process, the training epochs of the single-speaker and multi-speaker models were set to 50 and 100, respectively, to achieve better convergence.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Location**} & \multirow{2}{*}{**Bandwidth**} & \multicolumn{8}{c}{**Gap Length (ms)**} \\ \cline{3-8}  & & **20** & **40** & **50** & **100** & **200** & **500** & **1000** \\ \hline \multirow{2}{*}{1} & NB & 4.13 & 3.99 & 3.94 & 3.82 & 3.61 & 3.25 & 2.77 \\  & WB & 4.12 & 3.97 & 3.92 & 3.79 & 3.50 & 2.97 & 2.33 \\ \hline \multirow{2}{*}{2} & NB & 4.04 & 3.92 & 3.87 & 3.76 & 3.58 & 3.20 & 2.67 \\  & WB & 4.05 & 3.93 & 3.88 & 3.78 & 3.51 & 2.97 & 2.33 \\ \hline \multirow{2}{*}{3} & NB & 4.11 & 3.97 & 3.92 & 3.80 & 3.60 & 3.23 & 2.71 \\  & WB & 4.12 & 3.96 & 3.91 & 3.79 & 3.51 & 2.97 & 2.33 \\ \hline \multirow{2}{*}{4} & NB & 4.02 & 3.89 & 3.84 & 3.72 & 3.54 & 3.19 & 2.69 \\  & WB & 4.03 & 3.90 & 3.84 & 3.71 & 3.43 & 2.82 & 2.19 \\ \hline \multirow{2}{*}{5} & NB & 4.10 & 3.93 & 3.89 & 3.77 & 3.57 & 3.20 & 2.65 \\  & WB & 4.05 & 3.92 & 3.85 & 3.72 & 3.41 & 2.79 & 2.12 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Average MOSs of the inpainted speech with different gap lengths under different locations of the dropout layer.