In this work, a speaker embedding de-mixing approach is proposed . Instead of separating  two-speaker signal in signal space like speech source separation, the proposed approach separates different speaker properties from  two-speaker signal in embedding space. The proposed approach contains two steps. In step one, the clean speaker embeddings are learned and collected by a residual TDNN based network. In step two, the two-speaker signal and the embedding of one of the speakers are  input to a speaker embedding de-mixing network. The de-mixing network is trained to generate the embedding of the other speaker of the by reconstruction loss. Speaker identification accuracy on the de-mixed speaker embeddings is used to evaluate the quality of the obtained embeddings. Experiments are done in two kind of data: artificial augmented two-speaker data (TIMIT) and real world recording of two-speaker data (MC-WSJ). Six diffident speaker embedding de-mixing architectures are investigated. Comparing with the speaker identification accuracy on the clean speaker embeddings (98.5\%) , the obtained results show that one of the speaker embedding de-mixing architectures obtain close performance, reaching 96.9\% test accuracy on TIMIT when the SNR between the target speaker and interfering speaker is 5 dB. More surprisingly, we found choosing a simple subtraction as the embedding de-mixing function could obtain the second best performance, reaching 95.2\% test accuracy .
Separating different speaker properties from a multi-speaker environment is challenging . Instead of separating a two-speaker signal in signal space like speech source separation, a speaker embedding de-mixing approach is proposed. The proposed approach separates different speaker properties from a two-speaker signal in embedding space. The proposed approach contains two steps. In step one, the clean speaker embeddings are learned and collected by a residual TDNN based network. In step two, the two-speaker signal and the embedding of one of the speakers are both input to a speaker embedding de-mixing network. The de-mixing network is trained to generate the embedding of the other speaker by reconstruction loss. Speaker identification accuracy and the cosine similarity score between the clean embeddings and the de-mixed embeddings are used to evaluate the quality of the obtained embeddings. Experiments are done in two kind of data: artificial augmented two-speaker data (TIMIT) and real world recording of two-speaker data (MC-WSJ). Six different speaker embedding de-mixing architectures are investigated. Comparing with the performance on the clean speaker embeddings , the obtained results show that one of the proposed architectures obtained close performance, reaching 96.9\% identification accuracy and 0.89 cosine similarity .
R
In this work, a speaker embedding de-mixing approach is proposed
Separating different speaker properties from a multi-speaker environment is challenging
A
a
R
the proposed
a speaker embedding de-mixing approach is proposed. The proposed
A
a
A
both
D
of the
R
on the
and the cosine similarity score between the clean embeddings and the
R
speaker embeddings is
embeddings are
R
diffident
different
R
speaker identification accuracy
performance
D
(98.5\%)
R
speaker embedding de-mixing architectures obtain
proposed architectures obtained
R
test accuracy on TIMIT when the SNR between the target speaker and interfering speaker is 5 dB. More surprisingly, we found choosing a simple subtraction as the embedding de-mixing function could obtain the second best performance, reaching 95.2\% test accuracy
identification accuracy and 0.89 cosine similarity