Title: Counterpoint by Convolution
Authors: Cheng-Zhi Anna Huang
Tim Cooijmans
Adam Roberts
Aaron Courville
Douglas Eck
Venue: ICLR 2017
Keywords: Deep learning
Applications
Unsupervised Learning

Abstract: Machine learning models of music typically break down the task of composition into a chronological process, composing a piece of music in a single pass from beginning to end. On the contrary, human composers write music in a nonlinear fashion, scribbling motifs here and there, often revisiting choices previously made. We explore the use of blocked Gibbs sampling as an analogue to the human approach, and introduce Coconet, a convolutional neural network in the NADE family of generative models. Despite ostensibly sampling from the same distribution as the NADE ancestral sampling procedure, we find that a blocked Gibbs approach significantly improves sample quality. We provide evidence that this is due to some conditional distributions being poorly modeled. Moreover, we show that even the cheap approximate blocked Gibbs procedure from Yao et al. (2014) yields better samples than ancestral sampling. We demonstrate the versatility of our method on unconditioned polyphonic music generation.

TL;DR: NADE generative model of music, with new insights on sampling

Review discussion:
Will the authors release the code to github?
We started this work at Magenta[1] and as such the code will be checked into the Magenta github[2] soon.
However our (messy) research code is already available in our development repository[3].
[1] https://magenta.tensorflow.org
[2] https://github.com/tensorflow/magenta
[3] https://github.com/czhuang/magenta-autofill/tree/fixing/magenta/models/basic_autofill_cnn
- Is there a principled value for the number of sampling loops n in algorithm 1? If not, how did the authors typically chose n.
Also, in the conditioned rewriting task, how did the results change for varying n?
In particular, is the musical quality stable for large values of n?
- In section 7.3, a model that was trained on denoising is introduced. How does this model relate to the other models?
In particular, how was the context C chosen during training?
Edit:
- It would make sense to me to include a trivial baseline model in the human evaluation to better judge the significance of those results. Even a model that just randomly shuffles the time dimension would be a helpful sanity check for the quality of the AMT responses.
Thank you for your comments.  We have been improving the paper in response to comments from colleagues and the new version of the paper addresses some of your comments.  The new version puts greater emphasis on sampling, particularly on our main finding that blocked Gibbs sampling improves upon NADE ancestral sampling, and notably the approximate scheme from Yao et al is both faster and better in terms of sample quality. This is surprising as all three sampling procedures we discuss theoretically sample from the same distribution, and we conjecture (and tentatively confirm) that this is caused by some conditional distributions being poorly modeled. We evaluate sample quality primarily by likelihood under the model, supplemented with a new human evaluation study.
> Is there a principled value for the number of sampling loops n in algorithm 1? If not, how did the authors typically chose n.
Also, in the conditioned rewriting task, how did the results change for varying n?
In particular, is the musical quality stable for large values of n?
There is no principled way of choosing N, however Section 6.2 now discusses the convergence behavior of different Gibbs procedures.  Figure 2 shows results on how negative log-likelihood (NLL) of samples under the model improves with the length of the Gibbs chain.
In particular we see that all of them converge, which suggests that musical quality is indeed stable for large values of n.
Independent blocked Gibbs (due to Yao et al, see Section 5.2 for a description of the algorithm) is the preferred sampling procedure because it is among the sampling procedures that achieve the best log-likelihoods (see Table 2) and it is dramatically faster as it requires only one model evaluation per Gibbs step (as opposed to O(IT) model evaluations for the ancestral blocked Gibbs procedures (previously Algorithm 1)).  We show in Section 6.3 through human listening tests that samples from the independent Gibbs procedure are considered more musical than those from ancestral sampling (NADE).
> In Section 7.3, a model that was trained on denoising is introduced. How does this model relate to the other models?
In particular, how was the context C chosen during training?
For the denoising model, the context was chosen according to independent Bernoulli variables with inclusion probability 0.5.  The difference in this model is in how the input data is set up, in particular no masks are used: the variables not in the context C are randomly perturbed rather than masked, and the model does not know which variables constitute the context (i.e. no mask is fed in). This setup essentially requires the model to both identify which parts have been perturbed as well as how to correct them.
However, in the process of revising our paper to better bring out our main finding, we decided to omit the denoising variant.
> It would make sense to me to include a trivial baseline model in the human evaluation to better judge the significance of those results. Even a model that just randomly shuffles the time dimension would be a helpful sanity check for the quality of the AMT responses.
As a result of focusing more on comparing sampling procedures rather than model types, we have removed the previous human evaluation results on model types and added new results on comparing sampling procedures (see Section 6.3).  Here the baseline used is ancestral sampling (NADE), which is the conventional way of sampling from orderless NADEs (see Section 5.1 for a discussion).  In Section 6.4, we see Independent Gibbs outperform ancestral sampling (NADE) by a large margin on the human evaluation test.  This ordering was also found in the log-likelihood evaluations (see Table 2).
We share your concern on the reliability of Amazon Mechanical Turk.
However, our new human evaluation is stronger than the previous one.  Our previous human evaluation did not have a clear baseline.  Now, ancestral sampling (NADE) forms our baseline as this is the conventional way of sampling from NADE models.
Also, the performance margins between the models are much more pronounced than in the previous experiment.  This in our opinion lessens the need to compare to a trivial baseline.  We agree though that it would be a good sanity check, and will definitely include a trivial baseline if/when we redo the human evaluation.
Thank you again for spending the time to help us improve our work.
This paper proposed COCONET, which is a neural autoregressive model with convolution, to do music composition task. This paper also proposed to use blocked Gibbs sampling instead of the ancestral sampling of the original NADE model to generate better pieces of music. The experimental results showed that the NLL of COCONET is better than the other baselines and the human evaluation task by Amazon’s Mechanical Turk illustrated that the model can generate compelling music.
In general, I think the paper is good. Using NADE based model with convolution operations on music generation tasks and using blocked Gibbs sampling contains some kind of novelty.
However, the novelty of the paper is incremental, since the blocked Gibbs sampling for NADE model is already proposed by Yao et al., (2014) and the using NADE based model for music modeling has also been proposed by Boulanger-Lewandowski  et al., (2012).
Thank you for your review!  To clarify a few points:
There are two ways in which NADEs (Larochelle & Murray, 2011) are used in Boulanger-Lewandowski et al. (2012), both of which are different from our approach.  As a baseline, NADE was used to model distribution of chords/simultaneities, without considering temporal dynamics.
Then, as an alternative to RNN-RBMs, NADE was used to model the output distribution of RNNs, giving rise to RNN-NADEs.  Temporally, RNN-NADE factorizes sequences chronologically, and hence can only generate left to right.  As only one ordering is modeled, Gibbs sampling can not be used to improve sample quality.
In contrast, we adopt an orderless NADE (Uria et al., 2014) which is trained to learn all possible orders of factorization, making it possible to use Gibbs sampling to improve sample quality.
Furthermore, this approach frees music models from generating left to right, and can support tasks such as partial score completion that requires the versatility of conditioning on arbitrary context and to generate in any order.
Yao et al (2014) only expected independent blocked Gibbs to perform as well as ancestral sampling at best in terms of sample quality.  Their work focused on MNIST.  For the more intricate domain of music, we report the surprising observation that independent blocked Gibbs actually beats ancestral sampling.  We investigate why this might be the case.  Our finding softens the main drawback of autoregressive models, which is that sampling from them is slow. We believe our contribution is especially relevant given the recent popularity of autoregressive models (such as PixelCNN and WaveNet (van den Oord et al., 2016) etc.).
The paper presents a way to model the distribution of four-part Bach chorales using Convolutional Neural Networks.
Furthermore it addresses the task of artificial music generation by sampling from the model using blocked Gibbs sampling and shows
The CNN model for the distribution seems very appropriate for the data at hand.
Also the analysis of the proposed sampling schemes with the analogy between Gibbs sampling and human music composition are very interesting.
I am not too sure about the evaluation though. Since the reported likelihoods are not directly comparable to previous work, I have difficulties judging the quality of the quantitative results.  For the human evaluation I would like to see the data for the direct comparisons between the models. E.g. How did NADE vs. Bach perform.
Also I find the question: ‘what piece of music do you prefer’ a stronger test than the question ‘what piece is more musical to you’ because I don’t really know what ‘musical’ means to the AMT workers.
Finally, while I think the Bach Chorales are interesting musical pieces that deserve to be subject of the analysis but I find it hard to judge how well this modelling approach will transfer to other types of music which might have a very different data distribution.
Nevertheless, in conclusion, I believe this is an exciting model for an interesting task that produces non-trivial musical data.
In previous rounds of human evaluation, we did use your proposed question: "which piece do you prefer?".
However, users' stated reasons revealed that they based their ratings on (to us) irrelevant properties such as mood and synthesis details. The question "which piece is more musical?" reduced but did not eliminate this effect.
We have added more details on the human evaluation, including direct comparisons, in an appendix.
We are responding to your other questions in the response at the top of this page.
The paper tackles the task of music generation. They use an orderless NADE model for the task of "fill in the notes". Given a roll of T timesteps of pitches, they randomly mask out some pitches, and the model is trained to predict the missing notes. This follows how the orderless NADE model can be trained. During sampling, one normally follows an ancestral sampling procedure. For this, an ordering is defined over outputs, and one runs the model on the current input, samples one of the outputs according to the order, adds this output to the next input, and continues this procedure until all outputs have been sampled. The key point of the paper is that this is a bad sampling strategy.
Instead, they suggest the strategy of Yao et al. 2014, which uses a blocked Gibbs sampling approach. The blocked Gibbs strategy instead masks N inputs randomly and independently, samples them, and repeats this procedure. The point of this strategy is the make sure the sampling chain mixes well, which will happen for large N.
However, since the samples are independent, having a large N gives incoherent samples.
Thus, the authors follow an annealed schedule for N, making it smaller over time, which will eventually reduce to ancestral sampling (giving global structure to the sample). They conduct a variety of experiments involving both normal metrics and human evaluations, and find that this blocked Gibbs sampling outperforms other sampling procedures.
This is a well written paper - great job.
My main problem with the paper is that having read Uria and Yao, I don't know how much I have learned from this work in the context of this being an ICLR submission. If this was submitted to some computational music / art conference, this paper would be a clear accept.
However, for ICLR, I don't see enough novelty compared with previous works this builds upon. Orderless NADE is an established model. The blocked Gibbs sampling and annealing scheme are basically the exact same one used in Yao.
Thus, the main novelty of this paper is its application to the music domain, and finding that Yao's method works better for sampling music. This is a good contribution, but more tailored to those working in the music domain. If the authors found that these results also hold for other domains like images (e.g. on CIFAR / tiny Imagenet) and text (e.g. document generation), then I would change my mind and accept this paper for ICLR. Even just trying musical domains other than Bach chorales would be useful.
However, as it stands, the experiments are not convincing enough.
Thank you for your review.  We respond to your question above.
Thank you all for your reviews!
We share your concern regarding quantitative comparison to other works, and regarding the generality of our method.  We have looked into the datasets used in prior works, but found that their preprocessing severely degraded the musical structure.
For example, the temporal granularity used in Boulanger-Lewandowski et al. is too coarse: in several pieces, discarding all notes that do not have their onsets on the eighth-note grid results in sparse notes with long stretches of silence between them. The pieces become unrecognizable, and (worse) non-musical.  Downsampling or blurring does not work in symbolic music like it does in images: music is more similar to language, and using a coarse grid is analogous to removing words from a sentence and asking a model to learn this new distribution of broken sentences. This turns it into a different task, and makes qualitative judgement of samples not very meaningful.
We also considered applying our method to image data as suggested.
Unlike (symbolic) music which is discrete and intricately structured, the domain of images is smooth and forgiving of small errors. It is plausible that the NADE sampling approach generates fine images (indeed, that's what Yao et al. found), for reasons that don't carry over to our domain of interest, which is music.
The development of a larger-scale, more diverse and higher-quality MIDI music dataset is a major component of our ongoing project.  We nonetheless believe that the advances we show are of sufficient interest to justify publication at this time.

Meta-review:
role: PC
data: comment: This paper applies an existing idea (Yao's block Gibbs sampling of NADE) to a music model. There is also prior art for applying NADE to music. The main novel and interesting result is that block Gibbs sampling (an approximation) actually improves performance, highlighting problems with NADE.
 
 This work is borderline for inclusion. The paper is mainly an application of existing ideas. The implications of the interesting results could perhaps have been explored further for a paper at a meeting on learning representations.

Decision: Reject