Original: SpeakerGAN: Recognizing Speakers in New Languages with Generative Adversarial Networks
Revised: SpeakerGAN: Recognizing Speakers in New Languages with Generative Adversarial Networks

Original: Abstract
Revised: Abstract

Original: Verifying a person’s identity based on their voice, is a challenging, real-worldproblem in biometric security.
Revised: Verifying a person’s identity based on their voice is a challenging, real-worldproblem in biometric security.

Original: A crucial requirement of such speaker veriﬁcationsystems is to be domain robust.
Revised: A crucial requirement of such speaker veriﬁcationsystems is to be domain robust.

Original: Performance should not degrade even if speakersare talking in languages not seen during training.
Revised: Performance should not degrade even if speakersare talking in languages not seen during training.

Original: To this end, we propose to use Generative Adversarial Networks to adapt a  speaker embedding model to newlanguages using a small amount of unlabelled data.
Revised: To this end, we present a ﬂexibleand interpretable framework for learning domain invariant speaker embeddingsusing Generative Adversarial Networks.

Original: Our model is optimized end-toend, and veriﬁcation can be performed using simple cosine scoring. Additionally,we propose a novel objective for training the generator, that yields further improvements.
Revised: We combine adversarial training withan angular margin loss function, which encourages the speaker embedding modelto be discriminative by directly optimizing for cosine similarity between classes. We are able to beat a strong baseline system using a cosine distance classiﬁerand a simple score-averaging strategy.

Original: We show that the proposed unsupervised adversarial adaptation leadsto veriﬁcation performance that is competitive with state-of-the-art veriﬁcationsystems.
Revised: Our results also show that models withadversarial adaptation perform signiﬁcantly better than unadapted models.

Original: In an attempt to better understand the performance of our models, wequantitatively measure the degree of invariance induced by our proposed methodsusing Maximum Mean Discrepancy and Fréchet distances.
Revised: In anattempt to better understand this behavior, we quantitatively measure the degree ofinvariance induced by our proposed methods using Maximum Mean Discrepancyand Fréchet distances.

Revised: Our analysis shows that our proposed adversarial speakerembedding models signiﬁcantly reduce the distance between source and target datadistributions, while performing similarly on the former and better on the latter.

Original:  Introduction
Revised:  Introduction

Original: Text-Independent Speaker Veriﬁcation remains a challenging problem in the domain of biometricsecurity.
Revised: Text-Independent Speaker Veriﬁcation remains a challenging problem in the domain of biometricsecurity.

Original: Armed with the machinery of deep learning, veriﬁcation systems can now be deployedin the wild and are still capable of delivering robust performance.
Revised: Armed with the machinery of deep learning, veriﬁcation systems can now be deployedin the wild and are still capable of delivering robust performance.

Original: In the veriﬁcation community,situations wherein the test data is signiﬁcantly different from the data available during system trainingare referred to as - In the Wild.
Revised: In the veriﬁcation community,situations wherein the test data is signiﬁcantly different from the data available during system trainingare referred to as - In the Wild.

Original: For instance, the NIST-SRE 2016 evaluation data contains Cantoneseand Tagalog speakers (in-domain, target data), while most of the speakers in our training set aretalking in English (out-of-domain, source data).
Revised: For instance, the NIST-SRE 2016 evaluation data contains Cantoneseand Tagalog speakers (in-domain, target data), while most of the speakers in our training set aretalking in English (out-of-domain, source data).

Original: This distribution shift or mismatch between trainingand test data is an obstacle in several areas of pattern recognition and machine learning [1] and leadsto a degradation in system performance.
Revised: This distribution shift or mismatch between trainingand test data is an obstacle in several areas of pattern recognition and machine learning [1], and leadsto a degradation in system performance.

Original: The development biometric veriﬁcation system that performreliably in such conditions is critical for this technology be used safely and securely on a day-to-day basis
Revised: The development biometric veriﬁcation system that performreliably in such conditions is critical for this technology be used safely and securely on a day-to-daybasis.

Original: Deep neural networks (DNN) have revolutionized several areas of speech processing, and as such,are ideal candidates for learning discriminative speaker representations or embeddings [17, 7, 20, 2].
Revised: Deep neural networks (DNN) have revolutionized several areas of speech processing, and as such,are ideal candidates for learning discriminative speaker representations or embeddings [19, 9, 22, 3].

Original: Indeed, neural speaker embeddings have surpassed the performance of i-vectors [17, 3], especiallyon real world, in the wild data [14, 11].Arguably the most popular approach for learning speakerembeddings is to optimize the parameters of a DNN by minimizing the cross-entropy loss overspeakers in the training data.
Revised: Indeed, neural speaker embeddings have surpassed the performance of i-vectors [19, 5], especiallyon real world, in the wild data [16, 13].Arguably the most popular approach for learning speakerembeddings is to optimize the parameters of a DNN by minimizing the cross-entropy loss overspeakers in the training data.

Original: Cross-entropy is natural choice for identifying speakers, however itdoes not directly address the veriﬁcation task.
Revised: Cross-entropy is natural choice for identifying speakers, however itdoes not directly address the veriﬁcation task.

Original: As a consequence of not being optimized ‘end-toend’, the performance of cross-entropy speaker embeddings (X-vectors) is heavily dependent on apowerful classiﬁer to perform veriﬁcation.
Revised: As a consequence of not being optimized ‘end-toend’, the performance of cross-entropy speaker embeddings (X-vectors) is heavily dependent on a powerful classiﬁer to perform veriﬁcation.

Original: This dependence on a classiﬁer motivates the research anddevelopment of end-to-end systems.
Revised: This dependence on a classiﬁer motivates the research anddevelopment of end-to-end systems.

Original: We also believe that such systems can also beneﬁt in downstreamtasks that make use of speaker embeddings, such as speech recognition and synthesis.[-1x1,3x1,1x1,1x1, 256 3x1,1x1,1x1,3x1,1x1,1x1,3x1,1x1,xxxx AttentiveStatistics FC
-]
Revised: We also believe that such systems can also beneﬁt in downstreamtasks that make use of speaker embeddings, such as speech recognition and synthesis.[-1x1,3x1,1x1,1x1, 256 3x1,1x1,1x1,3x1,1x1,1x1,3x1,1x1,xxxx AttentiveStatistics FC
-]

Original: FC

Original: [Figure]

Original: Residual Blocks

Original: Conv1 3x23,

Original: Domain

Original: [Figure]

Original: Feature Extractor

Original: Speaker Classiﬁer

Original: [Figure]

Original: BCE Verifying a speaker’s identity is a challenging problem. Modern speaker veriﬁcation datasets like NIST-SRE 2016, add to this challenge by introducing a mismatch between the distributions of thetraining and test data.
Revised: Speakerveriﬁcation is a challenging problem, and modern veriﬁcation datasets like NIST-SRE 2016, addto this challenge by introducing a mismatch between the distributions of the training and test data.

Original: This phenomena is referred to as domain or covariate shift.
Revised: This phenomena is referred to as domain or covariate shift.

Original: In the case of NIST-SRE 2016, the test data consists of Cantonese and Tagalog speakers, whereas the vast majorityof training speakers are talking in English.
Revised: In the case of NIST-SRE 2016, the testdata consists of Cantonese and Tagalog speakers, whereas the vast majority of training speakers aretalking in English.

Original: NIST also provide a small amount of unlabelled, indomain, target data, that can be used to compensate for the domain shift.
Revised: NIST also provide a small amount of unlabelled, in-domain, target data, that canbe used to compensate for the domain shift.

Original: Most the domain adaptationtechniques that have been proposed for speaker veriﬁcation have been proposed on top of i-vectors orx-vectors.
Revised: Most the domain adaptation techniques that have beenproposed for speaker veriﬁcation have been proposed on top of i-vectors or x-vectors.

Original: In this  we present auniﬁed framework fordirectly learning domain-robust speaker embeddingsusing Generative Adversarial Networks (GAN).
Revised: In this work we present a framework for learning domain invariant speaker embeddings using Generative Adversarial Networks (GAN).

Original: We drawn inspiration from research in computervision, where GAN based  domain adaptation methods have been shown to be more powerful thanthe gradient reversal framework [18, 15, 16].
Revised: We drawn inspiration from research in computer vision,where GAN based unsupervised domain adaptation methods have been extremely successful [20,17, 18], and adapt these ideas for feature learning in a veriﬁcation setting.

Original: Both methods cast domain adaptation/invariance as anadversarial game - generate features or embeddings such that a discriminator cannot tell if they comefrom the source or target domain.
Revised: The basic idea is castdomain adaptation/invariance as an adversarial game - generate features or embeddings such that adiscriminator cannot tell if they come from the source or target domain.

Original: Unlike traditional GANs that work in high-dimensional spaces(e.g. natural images,speech), domain adaptation GANs operate in low-dimensional embedding space.
Revised: Unlike traditional GANs thatwork in high-dimensional spaces (e.g. natural images,speech), domain adaptation GANs operate inlow-dimensional embedding space.

Original: Keeping these constraints in mind, we propose a novel objective for updating the generator network,which we ﬁnd to work better than the conventional generative loss. The nature of the adversarial game makes training GAN models challenging.
Revised: We extend our recent work [2, 4] and propose a novel objectivefor updating the generator network. We ﬁnd that optimizing GAN models with this objective provesto be unstable, and propose to stabilize it by augmenting the discriminator with an auxiliary lossfunction.

Original: We found that a simpleway to stabilizethe training of our models was to make the GANs conditional.
Revised: This strategy also helped stabilize training for the conventional generator objective but wasnot strictly needed.

Original: Speciﬁcally wepropose to use a modiﬁed version of the Auxiliary Classiﬁer GAN (AuxGAN)[].
Revised: Additionally, we analyze the transformed source and target data distributions in order to gain furtherinsight regarding the performance of our method.

Original: We show that thisaddition makes our model robust and  we are able to use the same set of hyper-parameters to trainseveral GAN variants within our framework. In our experimentswe show thatall of the modelsoutperform the DANSE model, with some delivering comparable performance to a state-of-the-artx-vector system.
Revised: We measure distances between these distributionsusing Maximum Mean Discrepancy and Fréchet distances. From our analysis we see a that goodperformance in terms of distributional distance corresponds to good veriﬁcation performance. Ourspeaker veriﬁcation experiments show that the proposed adversarial speaker embedding frameworkdelivers robust performance, signiﬁcantly outperforming a strong i-vector baseline.

Original: Furthermore,  we are able tofuse different GAN models using simple score averagingto achieve state-of-the-art veriﬁcation performance.
Revised: Furthermore, byaveraging the scores of our different GAN models, we are able to achieve state-of-the-art results.

Original:  Models2.1 Feature Extractor (Generator)
Revised:  Models2.1 Feature Extractor (Generator)

Original: The ﬁrst step for learning discriminative speaker embeddings is to learn a mapping F ( X s ) −→ f ,f ∈ R D from a sequence of speech frames from speaker s to a D-dimensional feature vector f .
Revised: The ﬁrst step for learning discriminative speaker embeddings is to learn a mapping F ( X s ) −→ f ,f ∈ R D from a sequence of speech frames from speaker s to a D-dimensional feature vector f .

Original: F ( X )can be implemented using a variety of neural network architectures.
Revised: F ( X )can be implemented using a variety of neural network architectures.

Original: We design or feature extractorusing a residual network structure.
Revised: We design or feature extractorusing a residual network structure.

Original: We choose to model speech using 1-dimensional convolutionalﬁlters, owing to the fact that speech is translation invariant along the time-axis only.
Revised: We choose to model speech using 1-dimensional convolutionalﬁlters, owing to the fact that speech is translation invariant along the time-axis only.

Original: Following theresidual blocks we use a combination of self-attention and dense layers in order to represent inputaudio of arbitrary size by a ﬁxed-size vector, f .
Revised: Following theresidual blocks we use a combination of self-attention and dense layers in order to represent inputaudio of arbitrary size by a ﬁxed-size vector, f .

Original: Unlike traditional approaches, our proposed featureextractor is updated with an adversarial loss in addition to the standard task loss.
Revised: Unlike traditional approaches, our proposed featureextractor is updated with an adversarial loss in addition to the standard task loss.

Original: 2.2 Self-Attentive Speaker Statistics
Revised: 2.2 Self-Attentive Speaker Statistics

Original: Self-Attention models are an active area of research in the speaker veriﬁcation community.
Revised: Self-Attention models are an active area of research in the speaker veriﬁcation community.

Original: Intuitively,such models allow the network to focus on fragments of speech that are more speaker discriminative.
Revised: Intuitively,such models allow the network to focus on fragments of speech that are more speaker discriminative.

Original: The attention layers computes a scalar weight corresponding to each time-step t :
Revised: The attention layers computes a scalar weight corresponding to each time-step t :

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: These weights are then normalized, α t = softmax ( e t ) , to give them a probabilistic interpretation.
Revised: These weights are then normalized, α t = softmax ( e t ) , to give them a probabilistic interpretation.

Original: We use the attention model proposed in [22], which extends attention to the mean as well as standarddeviation:
Revised: We use the attention model proposed in [24], which extends attention to the mean as well as standarddeviation:

Original: [Equation]
Revised: [Equation]

Revised: t (2)

Revised: 1x1,3x1,1x1,1x1, 256 3x1,1x1,1x1,3x1,1x1,1x1,3x1,1x1,xxxx

Revised: AttentiveStatistics

Original: {+FC
+}
Revised: {+FC
+}

Revised: FC

Original: In this work we apply the use of self attention to convolutional feature maps, as indicated in Fig.
Revised: [Figure] Residual Blocks Conv1 3x23, [Figure]

Revised: Feature Extractor

Revised: Speaker Classiﬁer

Revised: [Figure]

Revised: BCE

Revised: [Equation]

Revised: In this work we apply the use of self attention to convolutional feature maps, as indicated in Fig.

Original: The last residual block outputs a tensor of size nBXnFxT , where nB is the batch size, nF is thenumber of ﬁlters and T is time.
Revised: The last residual block outputs a tensor of size nBXnFxT , where nB is the batch size, nF is thenumber of ﬁlters and T is time.

Original: The input to the attention layer, h t , is a nF dimensional vector.
Revised: The input to the attention layer, h t , is a nF dimensional vector.

Original: By using a self-attention model, we also equip our network with a more robust framework forprocessing inputs of arbitrary size than simple global averaging.
Revised: By using a self-attention model, we also equip our network with a more robust framework forprocessing inputs of arbitrary size than simple global averaging.

Original: This allows us simply forwardpropagate a recording through the network in order to extract speaker embeddings.
Revised: This allows us simply forwardpropagate a recording through the network in order to extract speaker embeddings.

Original: 2.3 Classiﬁer
Revised: 2.3 Classiﬁer

Original: The classiﬁer block, C ( f , θ y ) , is arguably the key component of the model, as it is responsible forlearning speaker discriminative features.
Revised: The classiﬁer block, C ( f , θ y ) , is arguably the key component of the model, as it is responsible forlearning speaker discriminative features.

Original: Recently, angular margin loss functions have been proposedas an alternative to contrastive loss functions for veriﬁcation tasks [8, 21].
Revised: Recently, angular margin loss functions have been proposedas an alternative to contrastive loss functions for veriﬁcation tasks [10, 23].

Original: The Additive Marginsoftmax (AM-softmax) loss function is one such algorithm with an intuitive interpretation.
Revised: The Additive Marginsoftmax (AM-softmax) loss function is one such algorithm with an intuitive interpretation.

Original: The losscomputes similarity between classes using cosine, and forces the similarity of the correct class to begreater than that of incorrect classes by a margin m .
Revised: The losscomputes similarity between classes using cosine, and forces the similarity of the correct class to begreater than that of incorrect classes by a margin m .

Original: [Equation]
Revised: [Equation]

Original: Where W T and f i are the normalized weight vector and speaker embedding respectively.
Revised: Where W T and f i are the normalized weight vector and speaker embedding respectively.

Original: The
Revised: The

Original: AM-softmax loss also adds a scale parameter s , which helps the model converge faster.
Revised: AM-softmax loss also adds a scale parameter s , which helps the model converge faster.

Original: We selectm = 0 .
Revised: We selectm = 0 .

Original: 6 and s = 30 for all our experiments.
Revised: 6 and s = 30 for all our experiments.

Original: 2.4 Domain Discriminator
Revised: 2.4 Domain Discriminator

Original: The domain discriminator D ( . ) is tasked with determining if embeddings come from the sourceor target domains, and is arguably the most important component of the model.
Revised: The domain discriminator D ( . ) is tasked with determining if embeddings come from the sourceor target domains, and is arguably the most important component of the model.

Original: In order to learndomain invariant features, we engage the domain discriminator in an adversarial game with the feaureextractor E ( . ) .
Revised: In order to learndomain invariant features, we engage the domain discriminator in an adversarial game with the feaureextractor E ( . ) .

Original: The domain discriminator consists of two fully connected layers followed by theoutput layer.
Revised: The domain discriminator consists of two fully connected layers followed by theoutput layer.

Original:  Domain Adversarial Speaker Embeddings
Revised:  Domain Adversarial Speaker Embeddings

Original: A key requirement for learning speaker embeddings that are domain invariant is to ﬁnd a balancebetween the task loss and the adversarial loss.
Revised: A crucial requirement for learning speaker embeddings that are domain invariant is to ﬁnd a balancebetween the task loss and the adversarial loss.

Original: The objective to learn a feature space wherein embeddings are speaker discriminative irrespective of the domain.
Revised: The objective to learn a feature space whereinembeddings are speaker discriminative irrespective of the domain.

Original: Key to achieving this is the domaindiscriminator D , which is trained using the Binary Cross-Entropy loss (BCE).
Revised: Key to achieving this is the domaindiscriminator D , which is trained using the Binary Cross-Entropy loss (BCE).

Original: [Equation]
Revised: [Equation]

Original: Where X s , X t represent source and target data respectively.
Revised: Where X s , X t represent source and target data respectively.

Original: E ( . ) is the feature extractor/generator.
Revised: E ( . ) is the feature extractor/generator.

Original: The adversarial game between D ( . ) and E ( . ) is given by:
Revised: The adversarial game between D ( . ) and E ( . ) is given by:

Original: [Equation]
Revised: [Equation]

Original: Equation (3) represents the most general form of the GAN game, and can be used to representdifferent adversarial frameworks depending on the choice of L advE
Revised: Equation (3) represents the most general form of the GAN game, and can be used to representdifferent adversarial frameworks depending on the choice of L advE

Original: Gradient Reversal: We obtain the gradient reversal framework by setting L advE = −L advD .
Revised: Gradient Reversal: We obtain the gradient reversal framework by setting L advE = −L advD .

Original: Gradient reversal optimizes the true minmax objective of the adversarial game.
Revised: Gradient reversal optimizes the true minmax objective of the adversarial game.

Original: However, thisobjective can become problematic, since the discriminator converges early during training and leadsto vanishing gradients.
Revised: However, thisobjective can become problematic, since the discriminator converges early during training and leadsto vanishing gradients.

Original: GAN: Rather than directly using the minimax loss, the standard way to train the generator is usingthe inverted label loss.
Revised: GAN: Rather than directly using the minimax loss, the standard way to train the generator is usingthe inverted label loss.

Original: The generator objective is given by:
Revised: The generator objective is given by:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: This splits the optimization into two independent objectives, one for the generator and one for thediscriminator.
Revised: This splits the optimization into two independent objectives, one for the generator and one for thediscriminator.

Original: This loss has the same ﬁxed-point properties as the minimax loss while providingstronger gradients to target mappings [18].
Revised: This loss has the same ﬁxed-point properties as the minimax loss while providingstronger gradients to target mappings [20].

Original: 3.1 Updating the Generator with Source Embeddings
Revised: 3.1 Updating the Generator with Source Embeddings

Original: In a typical GAN setting, the generator is trained only using fake data (with inverted labels).
Revised: In a typical GAN setting, the generator is trained only using fake data (with inverted labels).

Original: Thisstructure is also maintained in several adversarial domain adaptation algorithms.
Revised: Thisstructure is also maintained in several adversarial domain adaptation algorithms.

Original: However, in thecontext of this work we believe that updating the generator using both source and target data can bebeneﬁcial.
Revised: However, in thecontext of this work we believe that updating the generator using both source and target data can bebeneﬁcial.

Original: In this case, the generator loss simply inverts the discriminator loss of eq.
Revised: In this case, the generator loss simply inverts the discriminator loss of eq.

Original: [Equation]
Revised: [Equation]

Original: When using the proposed objective for training the generator, we are optimizing the true minimaxloss like in the gradient reversal approach.
Revised: When using the proposed objective for training the generator, we are optimizing the true minimaxloss like in the gradient reversal approach.

Original: Unfortunately, we found that optimizing this loss becomesunstable early during training.
Revised: Unfortunately, we found that optimizing this loss becomesunstable early during training.

Original: We found a simple approach to stabilize training for this model was toaugment the discriminator with an auxiliary loss function.
Revised: We found a simple approach to stabilize training for this model was toaugment the discriminator with an auxiliary loss function.

Original: This addition also makes training morestable when optimizing the standard generator objective.

Original: 3.2 Auxiliary Classiﬁer GAN
Revised: 3.2 Auxiliary Classiﬁer GAN

Original: The Auxiliary Classiﬁer GAN (AuxGAN) model augments the standard GAN framework with anauxiliary loss to perform conditional image generation [13].
Revised: The Auxiliary Classiﬁer GAN (AuxGAN) model augments the standard GAN framework with anauxiliary loss to perform conditional image generation [15].

Original: This approach aims to predict sideinformation (such as class labels), as opposed to feed the same information to the generator anddiscriminator.
Revised: This approach aims to predict sideinformation (such as class labels), as opposed to feed the same information to the generator and discriminator.

Original: In the context of this work, our goal is to use the prediction loss for regularization andrepresentation learning.
Revised: In the context of this work, our goal is to use the prediction loss for regularization andrepresentation learning.

Original: [Equation]
Revised: [Equation]

Original: Eq.
Revised: Eq.

Original: (6) is a modiﬁed version of the AuxGAN objective.
Revised: (9) is a modiﬁed version of the AuxGAN objective.

Original: In particular, the original formulation alsouses the auxiliary loss to train the generator as well (with fake data being assigned its own uniquelabel).
Revised: In particular, the original formulation alsouses the auxiliary loss to train the generator as well (with fake data being assigned its own uniquelabel).

Original: We found that the auxiliary loss was crucial for stabilizing L adv E ( X s , X t , D ) when usingthe formulation in eq.
Revised: We found that the auxiliary loss was crucial for stabilizing L adv E ( X s , X t , D ) when usingthe formulation in eq.

Original: In our experiments we found that the AuxGAN setup stabilizes modeltraining even when we use eq.
Revised: In our experiments we found that the AuxGAN setup stabilizes modeltraining even when we use eq.

Original: (3) as the generator objective, and leads to slightly better veriﬁcationperformance.
Revised: (7) as the generator objective, and leads to slightly better veriﬁcationperformance.

Original: In this setting only the discriminator is trained with the auxillary loss.
Revised: In this setting only the discriminator is trained with the auxillary loss.

Original: 3.3 GAN Variants
Revised: 3.3 GAN Variants

Original: Since their introduction, GANs have been one of the most researched topics in the deep learningcommunity.
Revised: Since their introduction, GANs have been one of the most researched topics in the deep learningcommunity.

Original: Several variations of the original formulation have been proposed, each with differentgenerative characteristics and stability issues.
Revised: Several variations of the original formulation have been proposed, each with differentgenerative characteristics and stability issues.

Original: In this work we explore three GAN variants in additionto the standard GAN - Least-Squares GAN [10], Auxiliary Classiﬁer GAN and Reletavistic GAN[6].
Revised: In this work we explore three GAN variants in additionto the standard GAN - Least-Squares GAN [12], Auxiliary Classiﬁer GAN and Relativistic GAN [8]. We use the standard average GAN variant of the Relativistic GAN model.

Original: These models differ in the structure of the discriminator network.
Revised: These models differ in thestructure of the discriminator network.

Original: We show that each varianttransforms the feature space in different way, will all the model showing mostly similar performance.
Revised: We show that each variant transforms the feature space indifferent way, will all the model showing mostly similar performance.

Original: Additionally we see that by fusing the performance of all GAN variants together through scoreaveraging we achieve the best overall performance.
Revised: Additionally we see that byfusing the performance of all GAN variants together through score averaging we achieve the bestoverall performance.

Original:  Experimental Setup
Revised:  Experimental Setup

Original: Training Data (Source): We used audio from previous NIST-SRE evaluations (2004-2010) and
Revised: Training Data (Source): We used audio from previous NIST-SRE evaluations (2004-2010) and

Original: Switchboard Cellular audio for training the proposed DANSE model as well as the x-vector andi-vector baseline systems.
Revised: Switchboard Cellular audio for training the proposed DANSE model as well as the x-vector andi-vector baseline systems.

Original: We also augment our data with noise and reverberation, as in [].
Revised: We also augment our data with noise and reverberation, as in [].

Original: We add128k noisy copies to the clean speech, ending up with ˜ 220k recordings in our training set.
Revised: We add128k noisy copies to the clean speech, ending up with ˜ 220k recordings in our training set.

Original: For DANSEmodel training we ﬁlter out speakers with less than 5 recordings, ending up with approximately6000 speakers, whereas the x-vector and i-vector systems were trained using the Kaldi recipe.
Revised: For DANSEmodel training we ﬁlter out speakers with less than 5 recordings, ending up with approximately6000 speakers, whereas the x-vector and i-vector systems were trained using the Kaldi recipe.

Original: Wenote that the vast majority of our training data consists of English speakers, and is recorded overtelephone/cellular channels.Model:
Revised: Wenote that the vast majority of our training data consists of English speakers, and is recorded overtelephone/cellular channels.

Original: In order to make a fair comparison, we use an identical network to the DANSE model.

Original:  The Embedding function/Generator , E ( . ) , consists of a 3 X 23 input convolutional layer, 4 residualblocks [3,4,6,3], an attentive statistics layer and two fully connected layers (512,512).
Revised: Model: The Embedding function/Generator , E ( . ) , consists of a 3 X 23 input convolutional layer, 4residual blocks [3,4,6,3], an attentive statistics layer and two fully connected layers (512,512).

Original: The classiﬁer , C ( . ) , module consists of a fully connected layer (64) and the AM-softmax output layer.
Revised: Theclassiﬁer , C ( . ) , module consists of a fully connected layer (64) and the AM-softmax output layer.

Original: The formeris the ﬁnal domain invariant speaker embedding extracted during evaluation.
Revised: The former is the ﬁnal domain invariant speaker embedding extracted during evaluation.

Original: Finally, the domaindiscriminator module consists of two fully connected layers (256,256) and a binary cross-entropyoutput layer.
Revised: Finally,the domain discriminator module consists of two fully connected layers (256,256) and a binarycross-entropy output layer.

Original: Exponential Linear Units (ELU) are used as non-linear activations for all layers of thenetwork.
Revised: Exponential Linear Units (ELU) are used as non-linear activations for alllayers of the network.

Original: Batch Normalization is used on all layers expect the attentive statistics layer.
Revised: Batch Normalization is used on all layers expect the attentive statistics layer.

Original: Optimization: We start by pre-training the Embedding function using standard cross-entropytraining.
Revised: Optimization: We start by pre-training the Embedding function using standard cross-entropytraining.

Original: Pre-training is carried out using the RMSprop optimizer with a learning rate (lr) of 0 .
Revised: Pre-training is carried out using the RMSprop optimizer with a learning rate (lr) of 0 .

Original: For training GAN based speaker embedding models we use different optimizers for training thethree networks (Embedding function,Classiﬁer, Discriminator).
Revised: For training GAN based speaker embedding models we use different optimizers for training thethree networks (Embedding function,Classiﬁer, Discriminator).

Original: The classiﬁer is optimized using
Revised: The classiﬁer is optimized using

Original: RMSprop with lr = 0 .
Revised: RMSprop with lr = 0 .

Original: 003 , while the domain classiﬁer and feature extractor are trained using SGDwith lr = 0 .
Revised: 003 , while the domain classiﬁer and feature extractor are trained using SGDwith lr = 0 .

Original: We were able to train all our GAN models using the same set of hyper-parameters.
Revised: We were able to train all our GAN models using the same set of hyper-parameters.

Original: We used performance on held out validation set to determine when to stop training.
Revised: We used performance on held out validation set to determine when to stop training.

Original: Data Sampling:
Revised: Data Sampling:

Original: We use an extremely simple approach for sampling data during training.
Revised: We use an extremely simple approach for sampling data during training.

Original: Wesample random chunks of audio (3-8 seconds) from each recording in the training set.
Revised: Wesample random chunks of audio (3-8 seconds) from each recording in the training set.

Original: We sampleeach recording 10 times to deﬁne an epoch.
Revised: We sampleeach recording 10 times to deﬁne an epoch.

Original: For each mini-batch of source data, we randomly sample(with repetition) a mini-batch from the unlabelled adaptation data for GAN training.
Revised: For each mini-batch of source data, we randomly sample(with repetition) a mini-batch from the unlabelled adaptation data for GAN training.

Original: Speaker Veriﬁcation:
Revised: Speaker Veriﬁcation:

Original: At test time we discard the domain discriminator branch of the model, as itis not needed for extracting embeddings.
Revised: At test time we discard the domain discriminator branch of the model, as itis not needed for extracting embeddings.

Original: Extraction is done by performing a forward pass on thefull recording, and using the 64-
Revised: Extraction is done by performing a forward pass on thefull recording, and using the 64-dimensional fc 3 layer as our speaker embeddings. Veriﬁcation trialsare scored using cosine distance. Veriﬁcation performance is reported in terms of Equal Error Rate(EER).

Original:  Results
Revised:  Results

Original: NIST-SRE 2016:
Revised: NIST-SRE 2016:

Original: Unlike previous years, The 2016 edition of the NIST-SRE introduced a challengingnew dataset containing Cantonese and Tagalog speakers.
Revised: Unlike previous years, The 2016 edition of the NIST-SRE introduced a challengingnew dataset containing Cantonese and Tagalog speakers.

Original: We use the Kaldi recipes for our baselinei-vector and x-vector systems.
Revised: We use the Kaldi recipes for our baselinei-vector and x-vector systems.

Original: We note that the x-vector baseline may be considered as state-of-the-artperformance on this dataset.
Revised: We note that the x-vector baseline may be considered as state-of-the-artperformance on this dataset.

Original: Adaptation Data (Target): 2722 unlabelled, target data recordings are provided to adapt veriﬁcationsystems.
Revised: Adaptation Data (Target): 2722 unlabelled, target data recordings are provided to adapt veriﬁcationsystems.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Tables 1 & 2. compares the performance of the different speaker representations on the NIST-SRE16task.
Revised: Tables 1 & 2. compares the performance of the different speaker representations on the NIST-SRE16task.

Original: The x-vector model produces the best results, however requires LDA based dimensionalityreduction and the PLDA classiﬁer.
Revised: Among the baseline systems the x-vector model produces the best results, however requires LDA based dimensionality reduction and the PLDA classiﬁer to produce its best result.

Original: The DASE model produces competitive results, showing similarperformance to the x-vector/PLDA system that does not use LDA. Our model also performs slightlybetter than the i-vector/PLDA system.
Revised: We seethat all of the GAN based models outperform gradient reversal by a large margin, but none of theindividual models are able to match the best x-vector system.

Original: We note that  the PLDA classiﬁer also requires signiﬁcanttuning and data augmentation. Furthermore, different PLDA implementations can lead to signiﬁcantlydifferent (worse) veriﬁcation performance.
Revised: Interestingly, we ﬁnd that we are ableto best this system by simply averaging the scores of our different GAN models. The FuseGANresults do not include the scores from the standard GAN model, although this does not affect the ﬁnalperformance signiﬁcantly.

Original:  Analysis
Revised:  Analysis

Original: One particularly interesting result from our experiments is the improvement we see through a simplescore averaging procedure.
Revised: One particularly interesting result from our experiments is the improvement we see through a simplescore averaging procedure.

Original: Our hypothesis is that the different discriminator objectives encourage thegenerator to cover different modes of the target data distribution.
Revised: Our hypothesis is that the different discriminator objectives encourage thegenerator to cover different modes of the target data distribution.

Original: This ﬁnding is consistent with GANapproaches that train multiple discriminators [12], although we do not train them simultaneously.
Revised: This ﬁnding is consistent with GANapproaches that train multiple discriminators [14], although we do not train them simultaneously.

Original: In
Revised: In

Original: Fig.
Revised: Fig.

Original: 2 we visualize the embedding spaces learned by our models using t-SNE [9].
Revised: 2 we visualize the embedding spaces learned by our models using t-SNE [11].

Original: While Gradient
Revised: While Gradient

Original: Reversal primarily appears to rotate the feature space, the transformations induced by the GANmodels is more pronounced.
Revised: Reversal primarily appears to rotate the feature space, the transformations induced by the GAN models is more pronounced.

Original: Crucially, we see that that the source domain speaker clusters appear toremain intact.
Revised: Crucially, we see that that the source domain speaker clusters appear toremain intact.

Original: This indicates that our models retain discriminative properties in the source domain, afact we verify experimentally.
Revised: This indicates that our models retain discriminative properties in the source domain, afact we verify experimentally.

Original: Maximum Mean Discrepancy (MMD) : is based on the idea that two distributions are identical ifand only if all their moments are identical [4].
Revised: Maximum Mean Discrepancy (MMD) : is based on the idea that two distributions are identical ifand only if all their moments are identical [6].

Original: A divergence can be deﬁned if we can measure how“different” the moments of the two distributions are.
Revised: A divergence can be deﬁned if we can measure how“different” the moments of the two distributions are.

Original: MMD is a method of efﬁciently doing this via the kernel trick:
Revised: MMD is a method of efﬁciently doing this viathe kernel trick:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: In order to quantitatively evaluate our models in terms of domain adaptation, we measure the
Revised: In order to quantitatively evaluate our models in terms of domain adaptation, we measure the

Original: Maximum Mean Discrepancy distance between a selection of source data and the unlabelled targetdata.
Revised: Maximum Mean Discrepancy distance between a selection of source data and the unlabelled targetdata.

Original: MMD is a standard distribution distance metric and has been applied in the context of domainadaptation [19].
Revised: MMD is a standard distribution distance metric and has been applied in the context of domainadaptation [21].

Original: Fréchet Distance: The Fréchet Inception Distance (ﬁd) is a popular approach for evaluating GANs,and has been shown to correlate well with human judgement of visual quality [5].
Revised: Fréchet Distance: The Fréchet Inception Distance (ﬁd) is a popular approach for evaluating GANs,and has been shown to correlate well with human judgement of visual quality [7].

Original: Instead of an
Revised: Instead of an

Original: Inception network, we extract embeddings from our gan models from the source and target data.
Revised: Inception network, we extract embeddings from our gan models from the source and target data.

Original: The
Revised: The

Original: Fréchet Distance between between the Gaussian ( m s , C s ) obtained from the source data distributionp s and the Gaussian ( m t , C t ) from the target data is given by:
Revised: Fréchet Distance between between the Gaussian ( m s , C s ) obtained from the source data distributionp s and the Gaussian ( m t , C t ) from the target data is given by:

Original: [Equation]
Revised: [Equation]

Original: Source Domain Speaker Veriﬁcation: We use the same source data used to compute the MMD and
Revised: Source Domain Speaker Veriﬁcation: We use the same source data used to compute the MMD and

Original: Fréchet Distance to construct a trial list for veriﬁcation.
Revised: Fréchet Distance to construct a trial list for veriﬁcation.

Original: The list consists of 2500 recordings and wescore them all versus all.
Revised: The list consists of 2500 recordings and wescore them all versus all.

Original: There are a total of 101,666 target and 6,145,834 non-target trials.
Revised: There are a total of 101,666 target and 6,145,834 non-target trials.

Original: From Fig.
Revised: From Fig.

Original: 3 we see that MMD and the Fréchet distance display similar trends.
Revised: 3 we see that MMD and the Fréchet distance display similar trends.

Original: Surprisingly we seethat Gradient Reversal only has a small effect on either metric, while the GAN models all have muchlower MMD and Fr’echet distances.
Revised: Surprisingly we seethat Gradient Reversal only has a small effect on either metric, while the GAN models all have muchlower MMD and Fr’echet distances.

Original: We note that the model using the novel generator objective showsthe lowest scores on both metrics.
Revised: We note that the model using the novel generator objective showsthe lowest scores on both metrics.

Original: The results on source domain speaker veriﬁcation also indicatethat our models remain discriminative in the source domain as well, with only a small degradationas compared to the unadapted model.
Revised: The results on source domain speaker veriﬁcation also indicatethat our models remain discriminative in the source domain as well, with only a small degradationas compared to the unadapted model.

Original: Interestingly, the Gradient Reversal model shows the bestperformance on this experiment albeit by a small margin. 
Revised: Interestingly, the Gradient Reversal model shows the bestperformance on this experiment albeit by a small margin. [Figure]

Original:  Conclusion
Revised:  Conclusion

Original: In this work we we presented a novel framework for learning domain-invariant speaker embeddingsusing GANs.
Revised: In this work we we presented a novel framework for learning domain-invariant speaker embeddingsusing GANs.

Original: By combining a powerful deep feature extractor, an end-to-end loss function and [Figure] most importantly, adversarial training we are able to learn extremely compact speaker embeddingsthat deliver robust veriﬁcation performance on challenging evaluation data.
Revised: By combining a powerful deep feature extractor, an end-to-end loss function andmost importantly, adversarial training we are able to learn extremely compact speaker embeddingsthat deliver robust veriﬁcation performance on challenging evaluation data.

Original: We showed that theproposed methods do reduce the domain mismatch between source and target data in terms of MMDand Fréchet distance.
Revised: We showed that theproposed methods do reduce the domain mismatch between source and target data in terms of MMDand Fréchet distance.

Original: Furthermore, we see that our methods adapt while maintaining their speakerdiscriminative nature in the source domain as well.
Revised: Furthermore, we see that our methods adapt while maintaining their speakerdiscriminative nature in the source domain as well.

Original: In future work we will experiment with other
Revised: In future work we will experiment with other

Original: GAN variants in an attempt to further improve performance.
Revised: GAN variants in an attempt to further improve performance