# An Electroglottograph Auxiliary Neural Network for Target Speaker Extraction
## 3 Materials and Methods
### Methods
#### 3.2.1 The Configuration of SpeakerBeam  
Our proposed models to deal with speech extraction problems were inspired by the SpeakerBeam network. The SpeakerBeam network is a combined network with multiple spectrogram amplitude inputs and spectrogram amplitude outputs. According to previous studies, the network can be divided into two parts: an auxiliary network that inputs the speech spectrograms of the target speaker's additional information and extracts the target speaker's features; and a main network that inputs mixed speech spectrograms and the target speaker's features and outputs estimated time-frequency masks. Finally, ISTFTs are performed using the enhanced amplitude spectrograms and the original phase spectrograms. Figure 2 shows the specific configuration of the SpeakerBeam network. The method used to extract features from the frequency domain of the target speaker's speech has worked well in previous studies. In this method, speech signals are transformed from the time domain into the two-dimensional frequency domain via STFTs. In our study, the vertical coordinate of a spectrogram was the feature dimension, which represented the feature vector of one frame. The horizontal coordinate was the frame dimension, and the time-domain sequence length divided by the window interval was rounded up to 173 columns. The length of the different speech samples was set to 1 s. The spectrograms were imported into the network after taking the modulus value. The estimated amplitude spectrograms of target speech signals were obtained by multiplying the amplitude spectrograms of mixed speech by the estimated mask values. The process was defined as follows:  
\[M=g(|Y|,|A|) \tag{4}\]  
\[\hat{S}=M\odot Y \tag{5}\]  
Figure 1: Diagrammatic sketches of SDR and SISDR, in which speech signals are indicated by vectors. where \(g\) stands for neural network's estimation of the mask of the STFT signal \(M\), \(Y\) and \(A\) represent the mixed speech and the extra information from the target speaker, respectively. \(|\cdot|\) is the magnitude of the signals, and \(\odot\) represents matrix multiplication of the mixed speech and binary masks. The horizontal coordinates of the spectrograms represented time, the vertical coordinates represented frequency, and the values of the coordinate points represented voice energy. In Figure 3, the deeper the color, the stronger the voice energy of the coordinate point. After the STFT transformation, the amplitudes of the different frequencies of speech were converted into logarithmic scales and the color sizes (amplitudes) were converted into decibels to form spectrograms. Then, the estimated speech was obtained by multiplying the mixed speech by the estimated masks. Neural networks that use additional information from target speakers are common modeling methods for solving acoustic problems. These networks can be expressed as follows:  
\[X_{k+1}=\sigma_{k}(L_{k}(X_{k},W,b)) \tag{6}\]  
where \(X_{k}\) is the input of the layer k, \(\sigma_{k}\) is the activation function, and \(L_{k}(X_{k},W,b)=WX+b\), where \(W\) is the weight and \(b\) is the bias vector of the layer. These networks can extract features from target speakers but may lack specificity for different features. The process of these methods is as follows:  
\[X_{k+1}=\sigma_{k}(\lambda\odot L_{k}(X_{k},W,b)) \tag{7}\]  
Figure 3: The spectrograms of the target speech, the mixed speech, mask and the estimated speech. The energy value is represented by the color. The deeper the color, the stronger the voice energy. Figure 2: The overall scheme of the SpeakerBeam network used to separate target speech from mixed speech. where \(\lambda\) is the weight vector of the features, which is determined by the target to be extracted by the network. SpeakerBeam [48] utilizes a sequence summarizing network with an attention mechanism. The attention mechanism can learn from different frames of speech and summarize the frequency information of a target speaker's speech.