The paper introduces SampleRNN, a hierarchical recurrent neural network model of raw audio. The model is trained end-to-end and evaluated using log-likelihood and by human judgement of unconditional samples, on three different datasets covering speech and music. This evaluation shows the proposed model to compare favourably to the baselines.  It is shown that the subsequence length used for truncated BPTT affects performance significantly, but interestingly, a subsequence length of 512 samples (~32 ms) is sufficient to get good results, even though the features of the data that are modelled span much longer timescales. This is an interesting and somewhat unintuitive result that I think warrants a bit more discussion.  The authors have attempted to reimplement WaveNet, an alternative model of raw audio that is fully convolutional. They were unable to reproduce the exact model architecture from the original paper, but have attempted to build an instance of the model with a receptive field of about 250ms that could be trained in a reasonable time using their computational resources, which is commendable.  The architecture of the Wavenet model is described in detail, but it found it challenging to find the same details for the proposed SampleRNN architecture (e.g. which value of "r" is used for the different tiers, how many units per layer, ...). I think a comparison in terms of computational cost, training time and number of parameters would also be very informative.  Surprisingly, Table 1 shows a vanilla RNN (LSTM) substantially outperforming this model in terms of likelihood, which is quite suspicious as LSTMs tend to have effective receptive fields of a few hundred timesteps at best. One would expect the much larger receptive field of the Wavenet model to be reflected in the likelihood scores to some extent. Similarly, Figure 3 shows the vanilla RNN outperforming the Wavenet reimplementation in human evaluation on the Blizzard dataset. This raises questions about the implementation of the latter. Some discussion about this result and whether the authors expected it or not would be very welcome.  Table 1 and Figure 4 also show the 2-tier SampleRNN outperforming the 3-tier model in terms of likelihood and human rating respectively, which is very counterintuitive as one would expect longer-range temporal correlations to be even more relevant for music than for speech. This is not discussed at all, I think it would be useful to comment on why this could be happening.  Overall, this an interesting attempt to tackle modelling very long sequences with long-range temporal correlations and the results are quite convincing, even if the same can't always be said of the comparison with the baselines. It would be interesting to see how the model performs for conditional generation, seeing as it can be more easily be objectively compared to models like Wavenet in that domain.    Other remarks:  - upsampling the output of the models is done with r separate linear projections. This choice of upsampling method is not motivated. Why not just use linear interpolation or nearest neighbour upsampling? What is the advantage of learning this operation? Don't the r linear projections end up learning largely the same thing, give or take some noise?  - The third paragraph of Section 2.1.1 indicates that 8-bit linear PCM was used. This is in contrast to Wavenet, for which an 8-bit mu-law encoding was used, and this supposedly improves the audio fidelity of the samples. Did you try this as well?  - Section 2.1 mentions the discretisation of the input and the use of a softmax to model this discretised input, without any reference to prior work that made the same observation. A reference is given in 2.1.1, but it should probably be moved up a bit to avoid giving the impression that this is a novel observation.
Output: en