HAPPIER: Hierarchical Polyphonic Music Generative RNN

1 INTRODUCTION . Drawing inspiration from aesthetic intuition and expressing it as music with specific domain knowledge , composing is a marvelous artistic process for musicians . Due to this uniqueness of composing , although not a new idea , automatic composing remains to be a challenging task . Recent advances in machine learning , especially in sequence generative models , have greatly enabled novel insights into algorithmic music generation and music data analysis ( Boulanger-Lewandowski et al. , 2012 ; Gu et al. , 2015 ; Chung et al. , 2015 ; Chu et al. , 2016 ; Engel et al. , 2017 ; Hadjeres et al. , 2017 ; Teng et al. , 2017 ; Jaques et al. , 2017 ; Yu & Varshney , 2017 ; Thickstun et al. , 2017 ; Roberts et al. , 2018 ) . However , recognizing underlying patterns beneath music across different levels of temporal abstraction while training , and remaining long-term consistency while composing , is still a fundamental challenge for automatic composition algorithms . Thus , most approaches suffer from the lack of coherent global structure of music generated : it lacks consistent theme or structure , and appears to be random and wandering ( Oord et al. , 2016 ) ( p. 8 ) . Designing hierarchical architectures to operate on different spatial and temporal resolution scales to ameliorate these problems in machine learning algorithms is not a new idea , and has achieved great success in various fields , including but not limited in the field of computer vision ( Lazebnik et al. , 2006 ) , reinforcement learning ( Dayan & Hinton , 1993 ; Kulkarni et al. , 2016 ) , and in the field of sequence modeling ( Hihi & Bengio , 1995 ; Koutnik et al. , 2014 ; Serban et al. , 2016 ; Mehri et al. , 2017 ) . We note that these kinds of hierarchical architectures correspond with the hierarchical nature of music : the composing of movements and phrases , the progression of chords , and the organization of notes within a measure lie on different temporal resolution levels , from the lowest to the highest . Although sharing some common traits , the patterns of them differ from each other significantly . We present an analogy in writing articles here as an illustration : when writing an article , a mature writer begins from organizing key ideas of each paragraph given the central idea of the whole article , then he arranges the progression of sentences within a paragraph conditioning on the main idea of the paragraph , and finally writes down words in each sentence given the idea of the sentence with respect to grammar rules . Although the whole article is a unified entity , the patterns and rules within each level , however , vary from each other . Thus , an efficient learning algorithm should be aware of this difference and is supposed to utilize it . Motivated by this intuition , we propose HAPPIER : a novel HierArchical PolyPhonic musIc gEnerative RNN model to generate music sequentially . For simplicity , we model music as two hierarchies : the higher level measure hierarchy and the lower level note hierarchy . We also assume that every measure in music is specified with a corresponding chord while composing . Under these assumptions , HAPPIER contains a higher-level LSTM ( Hochreiter & Schmidhuber , 1997 ) , a variant of RNN , learning correlations across measures and patterns for chord progressions , and a lower-level LSTM learning a conditional distribution over notes . The two hierarchies operate at different clock rates : the higher one updates every measure , while the lower one updates every unit duration . The higher level LSTM gives guidance over the lower level by projecting conditioning vectors to the lower one , meanwhile the lower level LSTM summarizes its cell states to the higher level LSTM once a measure in order to keep the latter one informed . The entire architecture is trained jointly end-to-end by back-propagation . HAPPIER , gained strength from hierarchical architectures , generates polyphonic music which maintains long-term dependencies , and performs better in listening tests compared to the state-of-the-art methods . Music samples generated by HAPPIER are provided in Appendix C . 2 RELATED WORKS . Designing automatic music generation algorithms always attracts great attention from researchers , even dating back to the 80s last century . Attempts include to compose with handcrafted constraints ( Ebcioglu , 1988 ) and to design machine learning algorithms , neural networks for instance ( Todd , 1989 ) . However , approaches merely with handcrafted rules generally have unsatisfactory performance because of the lack of variety in the music they generate . Thus , we adopt the learning approach in this work . Analyzing music data also gains great attention in the machine learning community especially recently ( Yu & Varshney , 2017 ; Thickstun et al. , 2017 ; Roberts et al. , 2018 ) . These works contribute much either in the perspective of datasets or learning hidden representations for music . There are two approaches to generate music with respect to the type of data representation : one approach generating raw audio ( Oord et al. , 2016 ; Engel et al. , 2017 ) , and the other composing music notes , e.g . in the form of MIDI or piano-roll ( Boulanger-Lewandowski et al. , 2012 ; Hadjeres et al. , 2017 ; Jaques et al. , 2017 ) . Our work belongs to the latter one . In this approach , most works consider monophonic composing ( Jaques et al. , 2017 ) , but advances have also been made recently in the more challenging polyphonic composing task ( Chu et al. , 2016 ) . In the light of data representation , our work is similar to the work of Yang et al . ( 2017 ) , both including simplified polyphonic MIDI representations of a chord track and a melody track . Recent advances in deep learning have enabled great progress in automatic music generation . Different network architectures and training algorithms have been designed , including Convolutional Neural Networks ( Oord et al. , 2016 ; Engel et al. , 2017 ; Yang et al. , 2017 ) , Recurrent Neural Networks ( RNN ) ( Boulanger-Lewandowski et al. , 2012 ; Chu et al. , 2016 ; Hadjeres et al. , 2017 ; Jaques et al. , 2017 ; Liang et al. , 2017 ; Lim et al. , 2017 ; Mehri et al. , 2017 ; Teng et al. , 2017 ; Ycart & Benetos , 2017 ) , and Generative Adversarial Nets ( Goodfellow et al. , 2014 ) ( Yang et al. , 2017 ) . However , as most of these works have pointed out themselves , generating music with coherent global structure still remains to be a major challenge for automatic composition algorithms . This difficulty may be primarily attributed to the inefficiency of models to learn long-term consistency in music , which is actually one of the fundamental challenges for most of sequence modeling tasks . Some of these works have made attempts to tackle this problem . Dating back to the 90s last century , the LSTM ( Hochreiter & Schmidhuber , 1997 ) architecture was proposed to encourage long-term memory in recurrent networks by presenting shortcut connections with gating functions . However , the melodies it generates still remain somewhat random ( Jaques et al. , 2017 ) ( p. 1 ) . Oord et al . ( 2016 ) designed dilated causal convolution architectures to enlarge the receptive fields of neurons in CNN to learn long-range consistencies . However , the generated samples still vary second-tosecond ( p. 8 ) . Jaques et al . ( 2017 ) regarded the generating process as a sequential decision making process , and adopted a reinforcement learning approach , combining data prior with domain specific knowledge as reinforcement learning rewards to enforce coherent global structures . However , this approach requires handcrafted knowledge of musical theories into the system , which may involve heavy efforts . Hadjeres et al . ( 2017 ) gained strength from pseudo-Gibbs sampling to iteratively tune generated chorales . However , since iterative approaches are much more slower than sequential approaches , this method has a rather low generating efficiency . This paper adopt a hierarchical approach to ameliorate the problem . Our work is related to SampleRNN ( Mehri et al. , 2017 ) , although their work does not consider automatic music composition problem . Similar to their approach , we use LSTM and different parts of our model run at different clock rates . Unlike their work , the higher hierarchy of our model operates on measures and chords , instead of on frames ( p. 2 ) , which are mere concatenations of inputs of the lower hierarchy . We remark that with the higher hierarchy operating on measures and chords , our model can explicitly utilize the knowledge that there are significant differences between the patterns of chord progressions and that of note organizations . Besides , if the number of hierarchies increases , the approach of simply concatenating the inputs of the lower hierarchies as the inputs of the higher ones may not be tractable , instead , some kind of dimension reduction is expected . In this way , we can regard our input of chords as a summarizing of the notes within each measure . Our approach also differs from SampleRNN in that summarizing paths from the lower hierarchy to the higher one is included . The benefits of these paths will be discussed and measured in Section 3 and Section 4 . Chu et al . ( 2016 ) also design hierarchical LSTM architectures for composing . However , they first generate melodies , and then accompany the melody with chords and percussion . Instead , we recommend inversely , which is delighted by the remark of Teng et al . ( 2017 ) ( p. 2 ) that ‘ This mimics how classical western Roman-numeral harmony is taught to beginners : only after one has the underlying chord sequence , can one explain the melody in terms of chord tones , passing tones , appoggiaturas , and so on. ’ Besides , although hierarchical , their approach does not have different parts of the model operating on various temporal resolution scales , and thus still can not efficiently model the structures of music at very different scales , which finally leads to the lack of hierarchical temporal structure in the music they generate ( Teng et al. , 2017 ) ( p. 1 ) . While although Teng et al . ( 2017 ) first generate chords , and then generate melody based on chords , their model is not end-to-end , and the subparts of their model also do not operate on different clock rates . 3 METHOD . In this paper , we propose a hierarchical LSTM ( Hochreiter & Schmidhuber , 1997 ) model HAPPIER for polyphonic music generation . For simplicity , we show a two-level hierarchical LSTM as an example of the model . The model can also be easily generalized to multiple hierarchies in the same way as the two-level model is constructed . Here , we represent polyphonic music as a combination of a chord track and a melody track , where every measure in the music is specified with a corresponding chord . An example of the representation is shown in Figure 1 . In previous works focusing on monophonic music generation e.g . ( Jaques et al. , 2017 ) , composition is generally modeled as a sequential process of generating notes . The process is modeled by a conditional distribution over the next note ni to generate given all the previous notes generated so far , a deep approximator f parametrized by θ , e.g . an LSTM , is used to approximate this distribution : p ( ni|n1 , n2 , .. , ni−1 ) ≈ f ( n1 , n2 , .. , ni ; θ ) ( 1 ) However , music composition is challenging to these approaches because music contains structures at very different scales , and thus melody generated by approaches with mere LSTM tends to be wandering and random , and lack of coherent global structures ( Jaques et al. , 2017 ) ( p. 1 ) . HAPPIER ameliorates the problem by designing end-to-end hierarchical architectures , including a measure level LSTM and a note level LSTM . The two hierarchies operate at different clock rates : the higher one updates every measure , while the lower one updates every unit duration . Our formulation of the problem is similar to Equation 1 , except that we also adopt a hierarchical representation of music . We denote the chord of the i-th measure ci , the j-th note in the i-th measure nij , the number of notes in the i-th measureNi , and the total number of measures in a piece of music N . The process is defined by jointly modeling Equation 2 via hierarchical LSTM : p ( nij |n11 , n12 , .. , ni ( j−1 ) ; c1 , c2 , .. , ci−1 ) p ( ci|n11 , n12 , .. , n ( i−1 ) Ni−1 ; c1 , c2 , .. , ci−1 ) ( 2 ) Figure 2 gives an overview of the architecture . Details of the architecture will be presented in Section 3.1 and Section 3.2 . In correspondence with this hierarchical architecture , we use the following way to represent music data . The way is inspired by the work of Hadjeres et al . ( 2017 ) , which aligns different tracks well for polyphonic music data representation . In this way , notes are represented by their MIDI pitches and time ticks denoting their beginning and holding ; while chords are represented by their pitch encodings and time ticks . Figure 3 below explains this representation method . One-hot vectors of these representations are used for network inputs . There are 129 types of notes and 79 types of chords in total in the Nottingham dataset ( Boulanger-Lewandowski et al. , 2012 ) . Thus , the prediction problem can be formulated into combinations of classification problems . For simplicity , we still denote notes as n and chords as c in the following sections , rather than separating their pitch and time tick representations . Readers may refer to Appendix B for details . To train the classifier , the sum of cross entropy losses of chords and notes between the predicted distribution and the actual target distribution is minimized , as shown below : min θ ∑ i∈ { 1,2 , .. , N } ( −ci log ( p̂ci |θ ) + ∑ j∈ { 1,2 , .. , Ni } ( −nij log ( p̂nij |θ ) ) ) ( 3 ) While during the generating process , we sample from the predicted distribution to get the next token given all the tokens generated so far . The process is repeated iteratively to generate complete music .

Summary: The authors propose a hierarchical model of symbolic music that takes explicit advantage of measures and chords to construct the hierarchy. Their model is very similar to SampleRNN (2-level RNN Autoregressive Model) but with an additional cross-entropy loss for chord labels at the higher level and a summarization connection passing back to the high level from the low-level at the end of each bar. They show that given monophonic music with chord labels their model is able to produce reasonably coherent chords and note samples, and improves the NLL over a low-level model alone.