# Voice Interaction Recognition Design in Real-Life Scenario Mobile Robot Applications
## 2 Human-Voice Interface System
### Speaker Separation System  
This paper utilized the speaker separation system [20] to achieve speaker-specific voice separation. The speaker separation system architecture shown in Figure 1 is divided into two main parts: (1) a speaker encoder, which is responsible for extracting the voice pattern characteristics of a specific speaker set by the user; and (2) a voice filter model, which is responsible for retaining and outputting the voice of a specific speaker in a noisy or multi-person environment. The two are trained separately; the details are as follows:  
(1) Speaker encoder  
The Speaker encoder extracts the embedding that represents the features of a particular speaker's voice pattern, which is called the d-vector. The speaker encoder architecture is a three-layer long short-term memory (LSTM) with generalized end-to-End loss (GE2E) [21]. The input is the Mel-inverted spectrum (MFC) [22] with reference audio conversion, having one frame every 1.6 s with each frameshift at 50%, which is equivalent to 0.8 s. The output is a 256-dimensional d-vector. The speaker encoder architecture is shown in Figure 2.  
(2) Voice Filter model  
This voice filter part is the main architecture of the system containing 8 layers of convolution layer, 1 layer of LSTM, and 2 layers of FC layer. The training and running process is shown in Figure 3. The input is a mix of audio and the d-vector. Before the mix enters the voice filter, it goes through short-time Fourier transform (SFTF), is converted  
Figure 1: Speaker separation system architecture diagram. Figure 2: Speaker-Encoder Architecture Diagram. into a spectrum map, and is input to the convolution layer. Meanwhile, the d-vector is used as an input for long-term memory (LSTM). There are two reasons for placing the d-vector in the long- and short-term memory (LSTM), but not in the convolutional layer as input. First, the d-vector already has good robustness to represent the vocal features of a particular speaker and does not need to be processed in the convolutional layer; and second, the convolutional layer assumes that time and frequency are aligned. Therefore, two completely different signals cannot be put together as inputs. As shown in Table 1, except for the last FC layer (FC) where the activation function is sigmoid, all the other layers are commonly used ReLU. Additionally presented in Table 1 are the parameters of each layer; the loss function uses mean square error (MSE). Table 1 shows the design of dilated convolutional layers to extract low-level acoustic features more efficiently. The time and frequency of the spectrum map in speech recognition were the same as the height and width of the image in image recognition. (3) Evaluation and Training in Voice Filter model \begin{table}
\begin{tabular}{l l l l l l} \hline \hline \multirow{2}{*}{**Layer**} & \multicolumn{2}{c}{**Kernel Size**} & \multicolumn{2}{c}{**Dilation**} & \multirow{2}{*}{**Filters/Nodes**} \\ \cline{2-2} \cline{5-6}  & **Time** & **Freq** & **Time** & & **Freq** \\ \hline Convolution Layer 1 & 1 & 7 & 1 & 1 & 64 \\ Convolution Layer 2 & 7 & 1 & 1 & 1 & 64 \\ Convolution Layer 3 & 5 & 5 & 1 & 1 & 64 \\ Convolution Layer 4 & 5 & 5 & 2 & 1 & 64 \\ Convolution Layer 5 & 5 & 5 & 4 & 1 & 64 \\ Convolution Layer 6 & 5 & 5 & 8 & 1 & 64 \\ Convolution Layer 7 & 5 & 5 & 16 & 1 & 64 \\ Convolution Layer 8 & 1 & 1 & 1 & 1 & 8 \\ \hline LSTM & & & & & 400 \\ \hline FC 1 & & & & & 600 \\ \hline FC 2 & & & & & 600 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters of voice filter network. Figure 3: Voice filter architecture diagram. The Libri speech dataset [23] was used in our voice filter model and its training flow is shown in Figure 4. First, before the model started training, all the parameters and training data of the model were set up in advance; the training data include the training set and test set. Training of the model began after setting one of the parameter settings to \"step\", which determines the time when the model weights are saved.