From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Reject

From: NIPS.cc/2018/Workshop/IRASL/Paper24/Authors
title: Response to reviewer 3
comment: We thank the reviewer for reading our contribution and providing feedback.

"* To my perspective, the proposed model stretches the definition of a variational autoencoder somewhat significantly.  There is still an encoder and decoder with a reconstruction loss and a loss based on the encoder representation.  However, the proposed model replaces the variational component of the loss with a discriminative loss function, either cross-entropy or triplet loss.  This is somewhat similar to the approach described in the cited Lamb, et al. paper "Discriminative regularization for generative models", but in that paper the KL-div loss is augmented with a discriminative loss rather than being replaced."

We agree with the reviewer in the sense the final model is different than VAEs. However, we reuse the VAE's framework, and the two loss components have somewhat the same goal as in the original setting. Our objective with this setting is to directly enforce desired properties in the latents whenever class labels are available, rather than employing non-informative priors and using the KL term simply as a regularization penalty.

"* The role of the short-duration window on the model or experiment is not clear.  In Section 4: "embeddings of each recording are obtained from 256 frames windows without overlap, and then averaged,".  It's not clear how this is different from mean-pooling.  It appears that this suggests that the model has access to the full utterance, but with a fixed convolutional window.  There doesn't seem to be anything specific about the model definition that requires or benefits from a short-duration window, it appears that any sequence summarizing representation could be used.

is not clearwhy short time.  ismn't it sjust mean pooling?"

We perform mean pooling only at test time. At train time the model only has access to segments of 256 frames. We use the speaker verification setting as test-bed for the general approach of enforcing class separability in the latents with supervision. We thus employ convolutional autoencoders commonly used for 2-dimensional data, and such setting requires fixed dimensionality of inputs.

* The discussion of "level of supervision" is more confusing than clarifying.  It is not clear why we should consider the proposed model as being "less" supervised than x-vectors, when both have access to a supervised label.  Also should cross-entropy loss be considered "more supervised" than triplet loss?

We agree with the reviewer in the sense that all settings (except the vanilla VAE) require class labels. Even the triplet-loss case rely on labels for triplets sampling. However, what we mean by level of supervision is how dependent training is of such information. The final weights of our model depend both on the loss signals provided by the reconstruction and the classification components, while x-vectors weights rely solely on the fully-supervised training. We argue the results indicate the increased level of supervision enforces domain-dependent representation.

Thank you for your comments, and we hope to have made your concerns more clear.

From: NIPS.cc/2018/Workshop/IRASL/Paper24/AnonReviewer3
title: VAE-inspired technique for speaker recognition
review: The authors propose an autoencoder model to learn a representation for speaker verification using short-duration analysis windows.  The variational loss term is replaced by a discriminative loss term.  The proposed approach works fairly well when compared to x-vectors.  However, a known adaptation technique provides much larger gains to an x-vector based system compared to the proposed.

* To my perspective, the proposed model stretches the definition of a variational autoencoder somewhat significantly.  There is still an encoder and decoder with a reconstruction loss and a loss based on the encoder representation.  However, the proposed model replaces the variational component of the loss with a discriminative loss function, either cross-entropy or triplet loss.  This is somewhat similar to the approach described in the cited Lamb, et al. paper "Discriminative regularization for generative models", but in that paper the KL-div loss is augmented with a discriminative loss rather than being replaced.

* The role of the short-duration window on the model or experiment is not clear.  In Section 4: "embeddings of each recording are obtained from 256 frames windows without overlap, and then averaged,".  It's not clear how this is different from mean-pooling.  It appears that this suggests that the model has access to the full utterance, but with a fixed convolutional window.  There doesn't seem to be anything specific about the model definition that requires or benefits from a short-duration window, it appears that any sequence summarizing representation could be used.

is not clearwhy short time.  ismn't it sjust mean pooling?

* The discussion of "level of supervision" is more confusing than clarifying.  It is not clear why we should consider the proposed model as being "less" supervised than x-vectors, when both have access to a supervised label.  Also should cross-entropy loss be considered "more supervised" than triplet loss?

* Why does the proposed technique not benefit from adapted PLDA to the extent that x-vectors do?  Perhaps more analysis would clarify this
rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper24/Authors
title: Response to reviewer 1
comment: We thank the reviewer for reading our contribution and for providing useful feedback.

Below, we quote the reviewer and address the comment right after:

"* Abstract : I think the authors meant "both triplet loss minimization and cross-entropy" instead of "both triplet loss minimization and speaker recognition"."

We thank the reviewer for pointing this out. By speaker recognition we mean cross-entropy minimization, as in a closed-set evaluation. However, in order to make it more clear, we will rephrase our abstract according to the suggestion.

"* The openslr.org link would be better put as a footnote.; * Please add "." at the end of the caption of Figure 3, Table 2 and Table 3.; * Section 2 would gain in readability if it's divided into two sub-sections (one for variational autoencoders and another for speaker verification) to separate the two subjects.; * In the table 1 caption : short short-duration --> short duration ?"

All the text corrections will be included in the final version in case of acceptance.

"* In the experimental section, the authors should have focused more on the "short utterance" aspect (since it appears in the paper's title).
The NIST SRE 2016 evaluation treated primarily the language mismatch problem (which explains the structure of Table 1) but a more duration-centered analysis of the proposed system's performance would have been better suited for this work. A duration-dependent visualization (figures 1,2,3) would have also been useful."

We apologize for the lack of clarity in our experiments description. All our experiments are performed in a short-duration setting, and our proposed systems only have access to speech segments of 256 frames (roughly 2.5 seconds). This will be made more clear in the final version of our manuscript.

We hope to have clarified the reviewer's concerns.

From: NIPS.cc/2018/Workshop/IRASL/Paper24/AnonReviewer1
title: VAE for short-duration speaker recognition 
review: In this paper, the authors propose a modified version of the variational autoencoder model to tackle the speaker recognition problem in the context of short-duration segments. A new multi-modal prior is learned by using a discriminative loss during the VAE training process. The proposed system is evaluated on the NIST SRE 2016 evaluation set and significant improvments are reported compared to x-vector and classical VAE systems.

The paper is well written and the theoretical/experimental sections are clear but a deeper analysis of the "short duration" aspect is missing. This work is a good step towards building VAE systems with better priors but more experiments are needed to better explain the gain of the proposed approach compared to the baselines.

* Abstract : I think the authors meant "both triplet loss minimization and cross-entropy" instead of "both triplet loss minimization and speaker recognition".

* The openslr.org link would be better put as a footnote.

* Please add "." at the end of the caption of Figure 3, Table 2 and Table 3.

* Section 2 would gain in readability if it's divided into two sub-sections (one for variational autoencoders and another for speaker verification) to separate the two subjects.

* In the table 1 caption : short short-duration --> short duration ?

* In the experimental section, the authors should have focused more on the "short utterance" aspect (since it appears in the paper's title).
The NIST SRE 2016 evaluation treated primarily the language mismatch problem (which explains the structure of Table 1) but a more duration-centered analysis of the proposed system's performance would have been better suited for this work. A duration-dependent visualization (figures 1,2,3) would have also been useful.

rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper24/Authors
title: Response to Reviewer 2
comment: We thank the reviewer for taking the time to read our paper and providing relevant feedback.

We address each of the reviewer's comments below:

"1. Even though the title says "short-duration", I don't see any analysis presented on short duration speaker recognition. I would highly recommend authors to break down the results on SRE 16 set based on duration. The eval set has the duration ranging from 10-60 sec. and can be obtained using the outputs of speech activity detection.  It would be very helpful for the audience to see the effectiveness of the proposed approach for the short-durations."

We apologize for the lack of clarity in our experiments description. We refer to the described setting as short-duration since our models only have access to segments of 256 frames (roughly 2.5 seconds). Our VAEs thus generate samples of shape [40, 256]. We do so by segmenting all of the train, enrollment, test, and unlabelled data with windows of size 256 with no overlap.

For the x-vectors case, we train the model with the Kaldi recipe. However, test is performed under the same conditions as in our proposed systems, i.e. the enrollment, test and unlabeled data are all segmented into short-duration chunks.

We made the updates in the manuscript to clarify these issues, and will submit an improved version in case of acceptance.

"2. Also, I feel the baseline system for x-vector is pretty weak. Especially, the EER for the Cantonese portion is very high. Even though the authors use almost similar training data and augmentation used in KALDI recipe. So the gains for VAEs in unadapted condition are probably much lower than reported. "

Training for the x-vectors is performed with the Kaldi recipe and results match those reported in literature for the full-length case. The mismatch in performance aptly observed by the reviewer is due to the evaluation scheme employed in short-duration chunks. We further highlight this difference in the current version of the manuscript.

"Minor comments:
Typos:
1. contructed —> constructed
2. Equal Error Rate in percentage 
3. mel—>Mel

References: 
Missing pp. in 1, 13, 15."

Corrected.

"Suggested to add this reference for NIST SRE 2016:
Sadjadi, S.O., Kheyrkhah, T., Tong, A., Greenberg, C., Reynolds, D., Singer, E., Mason, L., Hernandez-Cordero, J. (2017) The 2016 NIST Speaker Recognition Evaluation. Proc. Interspeech 2017, 1353-1357."

This reference was added.

We thank the reviewer once again for the comments and hope to have clarified the raised questions.

From: NIPS.cc/2018/Workshop/IRASL/Paper24/AnonReviewer2
title: VAE for speaker recognition 
review: In this work, authors have proposed two different strategies in variational autoencoders to provide prior knowledge implicitly. The VAE trained with proposed strategies is used to obtain low-dimensional speaker representation for the task of speaker recognition. Authors also compare VAE embeddings with TDNN based x-vectors for both domain adapted and unadapted PLDAs. The results are reported on major languages of NIST SRE 2016.

Major comments:

1. Even though the title says "short-duration", I don't see any analysis presented on short duration speaker recognition. I would highly recommend authors to break down the results on SRE 16 set based on duration. The eval set has the duration ranging from 10-60 sec. and can be obtained using the outputs of speech activity detection.  It would be very helpful for the audience to see the effectiveness of the proposed approach for the short-durations. 

2. Also, I feel the baseline system for x-vector is pretty weak. Especially, the EER for the Cantonese portion is very high. Even though the authors use almost similar training data and augmentation used in KALDI recipe. So the gains for VAEs in unadapted condition are probably much lower than reported. 

In summary, it is a good first step towards the use of VAEs with implicit priors in speaker recognition but in my view requires a little bit of more analysis and experimentation. 

Minor comments:
Typos:
1. contructed —> constructed
2. Equal Error Rate in percentage 
3. mel—>Mel

References: 
Missing pp. in 1, 13, 15.

Suggested to add this reference for NIST SRE 2016:
Sadjadi, S.O., Kheyrkhah, T., Tong, A., Greenberg, C., Reynolds, D., Singer, E., Mason, L., Hernandez-Cordero, J. (2017) The 2016 NIST Speaker Recognition Evaluation. Proc. Interspeech 2017, 1353-1357.

rating: 2: Marginally below acceptance threshold
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature