# Target Speaker Extraction by Fusing Voiceprint Features
## 1 Introduction  
Speech separation is an essential problem in human-computer interactions, the output of which provides clean audios for downstream tasks such as automatic speech recognition [1, 2, 3], speaker recognition [4], speaker diarization [5], etc. The existing speech separation models can be categorized into two types: speaker-independent and speaker-dependent. The speaker-independent models extract speech corresponding to different speakers from the mixing audio in an unsupervised way without any prior knowledge. They can be further classified into time-frequency-domain ones, such as deep clustering (DPCL) [6, 7], deep attractor network (DANet) [8] and permutation invariant training network (PITNet) [9], and time-domain ones, such as time-domain audio separation network (TasNet) [10] and fully convolutional time-domain audio separation network (Conv-TasNet) [11]. DPCL [6] first maps time-frequency (T-F) bins in the magnitude spectrograms into the embedding space and then uses a clustering algorithm to generate binary spectrogram masks.Then, these masks are applied on the original magnitude spectrograms to obtain masked spectrograms corresponding to a single speaker. DANet [8] forms a reference point (attractor) for each speaker in the embedding space, which is used as the center point for clustering. PITNet [9] uses PIT techniques that minimize reconstruction errors under minimum energy ranking to solve the permutation problem in the training stage. The time-frequency domain model calculates the short-time Fourier transform (STFT) to create a T-F representation of the mixing audio, which imposes an upper bound on separation performance. TasNet [10] replaced the STFT and inverse STFT with a trained encoder-decoder pair to remove the constraint between time precision and feature dimension size in the STFT. Conv-TasNet [11] uses stacked dilated 1-D convolutional blocks to replace the deep long short-term memory (LSTM) [12] networks in the TasNet for theseparation step, which increases the separation accuracy. The above methods extract the speech of different speakers from the mixing audio in an unsupervised way without any prior knowledge. Therefore, they are called speaker-independent methods. Nowadays, smart devices such as smart phones receive and recognize speech from the target speaker most of the time. Speech separation techniques towards these smart devices are more concerned about separating the target speaker's speech rather than all the speakers' speech. The speaker-dependent models focus on extracting a specific speaker's speech from the noisy audio based on speaker-dependent information. The task of speaker-dependent speech separation is also regarded as the target speaker extraction. A voiceprint is the sound spectrogram graph which reflects a speaker's voice characteristics. It provides useful knowledge for the speech separation models and helps to improve their performance consequently. For example, SpeakerBeam [13] uses an additional slice of speech of the target speaker to separate the target speaker from the interfering talkers. The method embeds a speaker-adaptive layer into a deep neural network (DNN) to adapt to the target speaker. VoiceFilter [14; 15] adopts the LSTM model to output the target speaker's speech. It uses a _speaker encoder_ to extract the target speaker's embedding and predicts the T-F bin mask corresponding to the target speaker using LSTM network. The target speaker extraction network (TENet) [16] accumulates the robust speaker embeddings over all the speech of a single speaker to get stable speaker characteristics in the training stage. It can achieve a good performance based on only a few anchors. Most target speaker extraction models adopt a speaker extraction module to predict a spectral mask based on the output of the voiceprint encoder. Both the speaker extraction module and the voiceprint encoder are implemented using deep networks and optimized jointly in the training stage.