From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Accept

From: NIPS.cc/2018/Workshop/IRASL/Paper49/Authors
title: Thank you for specific comments
comment: The paper can definitely be improved by providing results in multiple languages. 
Since there are multiple valid ways to define the reference system, we choose the one using the mfcc features of the raw audio signals and the phoneme sets and alignments given in the data set. Since the phonemes and alignments are already verified by the authors of the data set. 
The results can be improved more by including the reference system without clustering.

From: NIPS.cc/2018/Workshop/IRASL/Paper49/AnonReviewer1
title: Limited results
review: The paper presents an idea for clustering the output vectors of an encoder,  according to the weights by a soft alignment model and the correctness of the prediction of the decoder. Then, using these clustered vectors, is attempted a discovery of a phoneme set. 

The idea is interesting but the paper could benefit from providing results in both tonal and non-tonal languages. Additionally, the paper could benefit from showing the aligned sequences, before clustering, in order to justify the usage of the specific alignment model as a phoneme "polling" mechanism. 

Finally, the results could be improved more. 
rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper49/Authors
title: Answering questions
comment: Thank you very much for your detailed comments.

First of all, I wonder did you read the latest paper version I submit? The comments seems a bit off from what I had in the paper. 

Section 4.1: The number of unique phones is actually 47, and I changed that in the latest version. The number is determined by all unique phones appeared in TIMIT dictionary after eliminating stress markers. 

Section 4.2: There are 44 unique English phonemes, and 100 is approximately twice as many as number of English phonemes. So the cluster number is determined to be 100.

Section 5.1: I can include WER and CER information but I don't think it would be much different from the ones I had right now.

Section 5.2: We got rid of the tolerance values of calculating unique acoustic frames, and we treat each acoustic frame as one unique frame to simplify the analysis.

From: NIPS.cc/2018/Workshop/IRASL/Paper49/Authors
title: Answering questions of concern 
comment: Thank you very much for your comment. Here are some answers address your concerns.
For the "output phoneme", we have 100 clusters, and we group them down into a subset of TIMIT phones in order to simplify the analysis. It might be useful to use the cluster index to represent the output phonemes and calculate the entropy.
We just used one channel of attention in calculation because it seems to be the best way to find acoustic frames that were most important to each output character. 

Edited:
The set of phones learned by the algorithm contains 100 phones (the 100 clusters of the K-means clustering algorithm), but that we made the decision to map it down to a subset of the TIMIT phones in order to make it easier to understand the results (specifically, for the purpose of generating Table 1).  The original 100-cluster set could have been used for the analysis in Table 2; the results would certainly be different, but since we haven’t done the analysis yet, we can’t say what the results would be.

From: NIPS.cc/2018/Workshop/IRASL/Paper49/AnonReviewer3
title: Not so convincing experiments
review: This paper addresses the Listen, attend and spell based phone set discovery. The obtained phones are used for evaluation considering the metric in terms of sum of conditional entropy of the obtained phone set. It demonstrates that the discovered phone set is better than the original phone set.

Paper was well written and easy to follow, however, the experimental section can be improved.
It has moderate novelty: Contributions are clustering in the hidden node vectors to discover phone set and analysis of discovered phone set. Experiments is not so convincing.

Detailed comments:
Line 52: Reference is needed.
Section 4.1: Description of the data set needs improvement - e.g., how many no of dialects and what is size of train, dev and test sets. Describe the details of the phone set here instead of explaining in different parts example in section 5.2. Describe the steps to get the 46 unique phones. It is not clear whether data itself contains the 46 unique phones or it has been merged to obtain 46?

Section 4.2: line 131: what is original paper? Please provide reference?
Line 140: Reference for English phonemes and how many those phonemes?
How the number cluster are found to be 100?

Section 5.1: It would be interesting to see the WER and CER without clustering?

Section 5.2: it would be better to explain cluster to phone mapping in more detail such as how five trigraphs could result in one phone label?, how many ambiguous cases are manually assigned with the phones? and how those are assigned?
Please provide ref for mapped phone set.
It would be better if you provide ref/motivation for acoustic frame calculation. It is interesting to see how the entropy values are varied with different tolerance values?
rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper49/AnonReviewer2
title: Are results too limited?
review: I like the idea of this paper, and from the title I was expecting that they were going to cluster the sounds in some embedding space and define some new "phonemes."

The paper does part of this, but then as far as I can tell the "output phonemes" are only allowed to be a subset of the original TIMIT phones. It's not clear what we are to learn from this subset.

The key technical assumption of the paper seems to be the way that attention is used to pick which phoneme matters. Does this really give the right result, even though there are multiple channels of attention (at least in the original LAS paper... I'm not sure in this implementation.)

Given that the results do not beat the reference set, I can't argue for acceptance. 
rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct