Task: Language Identification
Input (Spanish, English): This submission proposes to learn the word decomposition, or word to sub-word sequence mapping jointly with the attention based sequence-to-sequence model. A particular feature of this approach is that the decomposition is not static, instead, it also conditions on the acoustic input, and the mapping is probabilistic, i.e., one word may map to multiple sub-word sequences. The authors argue that the dynamic decomposition approach can more naturally reflect the acoustic pattern. Interestingly, the motivation behind this approach is analogous to learning the pronunciation mixture model for HMM based speech recognition, where the probabilistic mapping from a word to its pronunciations also conditions on the acoustic input, e.g.,  I. McGraw, I. Badr, and J. Glass, "Learning lexicons form speech using a pronunciation mixture model," in IEEE Transactions on Audio, Speech, and Language Processing, 2013  L. Lu, A. Ghoshal, S. Renals, "Acoustic data-driven pronunciation lexicon for large vocabulary speech recognition", in Proc. ASRU   R. Singh, B. Raj, and R. Stern, "Automatic generation of subword units for speech recognition systems,"  in IEEE Transactions on Speech and Audio Processing, 2002  It would be interesting to put this work in the context by linking it to some previous works in the HMM framework.  Overall, the paper is well written, and it is theoretically convincing. The experimental study could be more solid, e.g., it is reasonable to have a word-level baseline, as the proposed approach lies in between the character-level and word-level systems. the vocabulary size of the WSJ si284 dataset is 20K at maximum, which is not very large for the softmax layer, and it is a closed vocabulary task. I guess the word-level system may be also competitive to the numbers reported in this paper. Furthermore, can you explain what is the computational bottleneck of the proposed approach? You downsampled the data by the factor of 4 using an RNN, and it still took around 5 days to converge. To me, it is a bit expensive, especially given that you only take one sample when computing the gradient. Table 2 is a little bit misleading, as CTC with language model and seq2seq with a language model model from Bahdanau et al. is much closer to the best number reported in this Table 2, while you may only get a very small improvement using a language model. Finally, "O(5) days to converge" sounds a bit odd to me.
Output (English): en