This paper discusses a text-to-speech system which is based on a convolutional attentive seq2seq architecture.   It covers experiments on a few datasets, testing the model's ability to handle increasing numbers of speakers. By and large, this is a "system" paper  - it mostly describes the successful application of many different existing ideas to an important problem (with some exceptions, e.g. the novel method of enforcing monotonic alignments during inference).   In this type of paper, I typically am most interested in hearing about *why* a particular design choice was made, what alternatives were tried, and how different ideas worked.   This paper is lacking in this regard  - I frequently was left looking for more insight into the particular system that was designed.   Beyond that, I think more detailed description of the system would be necessary in order to reimplement it suitably  (another important potential takeaway for a "system" paper).   Separately, I the thousands-of-speakers results are just not that impressive  - a MOS of 2 is not really useable in the real-world.   For that reason, I think it's a bit disingenuous to sell this system as "2000-Speaker Neural Text-to-Speech". For the above reasons, I'm giving the paper a "marginally above" rating.   If the authors provide improved insight, discussion of system specifics, and experiments, I'd be open to raising my review.   Below, I give some specific questions and suggestions that could be addressed in future drafts. - It might be worth giving a sentence or two defining the TTS problem  - the paper is written assuming background knowledge about the problem setting, including different possible input sources, what a vocoder is, etc.   The ICLR community at large may not have this domain-specific knowledge. - Why "softsign" and not tanh?   Seems like an unusual choice. - What do the "c" and "2c" in Figure 2a denote? - Why scale (h_k + h_e) by \sqrt{0.5} when computing the attention value vectors? - "An L1 loss is computed using the output spectrograms"  I assume you mean the predicted and target spectrograms are compared via an L1 loss.   Why L1? - In Vaswani et al., it was shown that a learned positional encoding worked about as well as the sinusoidal position encodings despite being potentially more flexible/less "hand-designed" for machine translation.   Did you also try this for TTS?   Any insight? - Some questions about monotonic attention: Did you use the training-time "soft" monotonic attention algorithm from Raffel et al. during training and inference, or did you use the "hard" monotonic attention at inference time?   IIUC the "soft" algorithm doesn't actually force strict monotonicity.   You wrote "monotonic attention results in the model frequently mumbling words",  can you provide evidence/examples of this?   Why do you think this happens?   The monotonic attention approach seems more principled than post-hoc limiting softmax attention to be monotonic,  why do you think it didn't work as well? - I can't find an actual reference to what you mean by a "wavenet vocoder".   The original wavenet paper describes an autoregressive model for waveform generation.   In order to use it as a vocoder, you'd have to do conditioning in some way.   How?   What was the structure of the wavenet you used?    Why?   These details appear to be missing.   All you write is the sentence (which seems to end without a period)  "In the WaveNet vocoder, we use mel-scale spectrograms from the decoder to condition a Wavenet, which was trained separated". - Can you provide examples of the mispronunciations etc. which were measured for Table 1?   Was the evaluation of each attention mechanism done blindly? - The 2.07 MOS figure produced for tacotron seems extremely low,  and seems to indicate that something went wrong or that insufficient care was taken to report this baseline.   How did you adapt tacotron (which as I understand is a single-speaker model) to the multi-speaker setting? - Table 3 begs the question of whether Deep Voice 3 can outperform Deep Voice 2 when using a wavenet vocoder on VCTK (or improve upon the poor 2.09 MOS score reported).   Why wasn't this experiment run? - The paragraph and appendix about deploying at scale is interesting and impressive,  but seems a bit out of place  - it probably makes more sense to include this information in a separate "systems" paper.