Encoding Musical Style with Transformer Autoencoders
Authors: Kristy Choi
Curtis Hawthorne
Ian Simon
Monica Dinculescu
Jesse Engel
Venue:  (2020)
Keywords: music generation
sequence-to-sequence model
controllable generation

Abstract: We consider the problem of learning high-level controls over the global structure of sequence generation, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance.  We show it is possible to combine this global embedding with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and  and melody. Empirically, we demonstrate the effectiveness of our method on a variety of music generation tasks on the MAESTRO dataset and an internal, 10,000+ hour dataset of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to relevant baselines.

Under review as a conference paper at ICLR 2020
ENCODING MUSICAL STYLE WITH TRANSFORMER AUTOENCODERS
Anonymous authors Paper under double-blind review
ABSTRACT
We consider the problem of learning high-level controls over the global structure of sequence generation, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer au- toencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global embedding with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and and melody. Empirically, we demonstrate the effectiveness of our method on variety music generation tasks on the MAESTRO dataset and an internal dataset with 10,000+ hours of piano performances, where achieve improvements in terms of log-likeliine -igliiielhoodd and mcan listening scores as compared to relevant baselines.
INTRODUCTION
There has been significant progress in generative modeling, particularly with respect to creative ap- plications such art and music (Oord al. 2016 Engel al. (20176 |l & 2017 Huang al.)
2019a Payne 2019). As the number of generative applications increase, it becomes increasingly important to consider how users can interact with such systems, particularly when the generative model functions as tool in their creative process (Engel et al. 2017a Gillick et al.) 2019) To this end, we consider how one can learn high-level controls over the global structure of generated sam-
ple. We focus symbolic music generation, where Music Transformer (Huang et al. (019h) is the current state-of-the-ars generating high-quality samples that span over a minute length.
The challenge in controllable sequence generation is the fact that Transformers (Vaswani et al.) 2017) and their variants excel as language models or in pequence-to-sqqueeee tasks such as translation, but it is less clear to how they can: (1) learn and (2) incorporate global conditioning information at inference time. This contrasts with traditional generative models for images such as the variational autoencoder (VAE) (Kingma & Welling 2013) generative adversarial network (GAN) (Goodfel-
low et al.] 2014) which typically incorprate global conditioning as part of their training procedure
(Sohn et al. 2015 Sønderby al. 2016 [sola al. 2017 Van den Oord et al., 2016).
In this work, we introduce the Transformer autoencoder, where we aggregate encodings across time obtain holistic representation of the performance style. We show that this learned global rep- resentation can be incorporated with other forms of structural conditioning in two ways. First, we show that given a performance, our model can generate performances that are similar in style the provided input. Then, we explore different methods to combine melody and performance represen- tations to harmonize a melody in the style of the given performance. In both cases, we show that combining both global and fine-scale encodings of the musical performance allows us to gain better control of generation, separately manipulating both the style and melody of the resulting sample.
Empirically, we evaluate our model on two datasets: the publicly-available MAESTRO (Hawthorne ct al. 2019) dataset, and an internal dataset of piano performances transcribed from 10,000+ hours of audio (Anonymous for review). We find that the Transformer autoencoder is able to generate not only performances that sound similar to the input, but also accompaniments of melodies that follow a given style, as shown through both quantitative and qualitative experiments well as user listening study. In particular, we demonstrate that our model is capable of adapting to a particular musical style even in the case where we have one single input performance.

---

Under review as a conference paper at ICLR 2020
(ophional)
JOMY
Melddy
temporal compresiles
Performance
Performance
I..] [..]
[embedding]
(MID)
(aptions)
Meledy
Molody
I.. 1.I.1.1.
(M4D))
Encoder
Figure 1: A flowchart of the Transformer autoencoder. We first transcribe the wav data files into MIDI using the Onsets and Frames framework, then encode them into performance representations to use ipput. The output of the performance encoder is aggregated across time and (optionally) combined with a melody embedding to produce a representation of the entire performance, which then used by the Transformer decoder at inference time.
2
PRELIMINARIES
2.1 DATA REPRESENTATION FOR MUSIC GENERATION
The MAESTRO (Hawthorne et al. 2019) dataset consists of over 1,100 classical piano perfor- mances, where each piece is represented MIDI file. The internal performance dataset consists of over 10,000 hours piano performances transcribed from audio (Anonymous for review). In both cases, represent music as sequence of discrete tokens, effectively formulating the gener- ation task as a language modeling problem. The performances are encoded using the vocabulary as described in (Oore al. 2018), which captures expressive dynamics and timing. This performance encoding vocabulary consists of 128 note_on events, 128 note.of events, 100 time-shife events representing time shifts in 10ms increments from 10ms 1s, and 32 quantized velocity bins representing the velocity at which the 128 note on events were played.
2.2 MUSIC TRANSFORMER
We build our Transformer autoencoder from Music Transformer, a state-of thef-hhh-art generative model that is capable of generating music with long-term coherence (Huang al.) (019b)). While the origi- nal Transformer uses a self-attention mechanism that operates over absolute positional encodings of each token in given sequence (Vaswani al. 2017). Music Transformer replaces this with relative attention (Shaw et al. 20188), which allows the model to keep better track of regularity based on event orderings and periodicity in the performance. Huang et (2019b) propose novel algorithm for implementing relative self-attention that is significantly more memory-efficient, enabling the model generate musical sequences over minute length. For more details regarding the self attentinn mechanism and Transformers, we refer the reader to Vasswani et al.) 2017: Parmar et al., 2018).
3
CONDITIONAL GENERATION WITH THE TRANSFORMER AUTOENCODER
3.1 MODEL ARCHITECTRRE
We leverage the standard encoder and decoder stacks of the Transformer as foundation for our model, with minor modifications that we outline below.
Transformer Encoder: For both the performance and melody encoder networks, we use the Trans- former's stack of 6 layers which are each comprised of a: (1) multi-head relative attention mech- anism; and a (2) position-wise fully-connected feed-forward network. The performance encoder takes input the event-based performance encoding of input performance, while the melody encoder learns an encoding of the melody which has been extracted from the input performance. Depending on the music generation task, which elaborate upon in Section 3.2, the encoder out- put(s) are fed into the Transformer decoder. Figure describes the way which the encoder and decoder networks are composed together.
2

---

Under review as a conference paper at ICLR 2020
Transformer Decoder: The decoder shares the same structure the encoder network, but with an additional multi-head attention layer over the encoder outputs. At each step of generation, the decoder takes in the output of the encoder, as well as each new token that previously generated.
The model is trained end-to-end with maximum likelihood. That is, for a given sequence a of length n, we maximize log (0 Ei=1 log po (x; with respect to the model parameters A.
3.2 CONDITIONING MECHANISM
Performance Conditioning and Bottleneck For this task, we aim to generate samples that sound "similar" conditioning input performance. We incorporate a bottleneck in the output of the Transformer encoder in order to prevent the model from simply memorizing the input (Baldi) 2012). Thus, as shown in Figure [ we rrean-aggregate the performance embedding across the time dimen- sion in order to learn global representation of style. This mean- reea--eerorrance embedding is then fed into the autoregressive decoder, where the decoder attends to this global representation in order to predict the appropriate target. Although this bottleneck may be undesirable in sequence transduction tasks where the input and output sequences differ (e.g. translation). we find that works well in our setting where we require the generated samples to be similar in style to the input sequence.
Melody & Performance Conditioning: Next, we synthesize any given melody in the style of a different performance. Although the setup shares similarities to that of the melody conditioning problem in (Huang et al. 2019b). we note that we also provide a conditioning performance signal, which makes the generation task more challenging. During training, we follow an internal proce- dure to extract melodies from performances the training set, quantize the melody to a 100ms grid, and encode it as sequence of tokens that uses different vocabulary than the performance repre- sentation. We then use two distinct Transformer encoders (each with the same architecture) as in Section 3.1 to separately encode the melody and performance inputs. The melody and performance embeddings are combined to use as input to the decoder.
We explore various ways of combining the intermediate representations: (I) sum, where add the performance and melody embeddings together: (2) concatenate where we concatenate the two embeddings separated with a stop token; and (3) tile, where we tile the performance embedding across every dimension of time in the melody encoding. In all three cases, we work with the mean- aggregated representation of the input performance. We find that different approaches work better than others on some dataets, point which we elaborate upon in Section 5
Input Perturbation In order to encourage the encoded performance representations to generalize across various melodies, keys, and tempos, we draw inspiration from the denoising autoencoder (Vincent et al. 2008) as a means to regularize the model. For every target performance from which extract the input melody, we provide the model with a perturbed version of the input performance as the conditioning signal. We allow this "noisy" performance to vary across two axes of variation: (1) pitch, where artificially shift the overall pitch either down or up by 6 semitones; and (2) t time, where we stretch the timing of the performance by at most 5%. In our experiments, we find that this augmentation procedure leads samples that sound more pleasing (Oore et al. 2018). We provide further details on the augmentation procedure in Appendix A.
4 SIMILARITY EVALUATION ON PERFORMANCE FEATURES
Although a variety of different metrics have been proposed to quantify both the quality (Engel et al., 2019) and similarity of musical performances relative to one another (Yang & Lerch] 2018 0 Hung ct al. 2019), the development of a proper metric to measure such characteristics music generation remains an open question. Therefore, we draw inspiration from (Yang & Lerch 2018) to capture the style of given performance based its the pitch- and thythm-related features using 8 fcatures:
1. Note Density (ND): The note density refers to the average number of notes per second in performance: a higher note density often indicates a fast-moving piece, while a lower note density correlates with softer, slower pieces. This feature is a good indicator for rhythm.
2. Pitch Range (PR): The pitch range denotes the difference between the highest and lowest semitones (MIDI pitches) in a given phrase.
3

---

Under review as a conference paper at ICLR 2020
3. Mean Pitch (MP) / Variation of Pitch (VP): Similar in vein to the pitch range (PR), the average and overall variation of pitch in a musical performance captures whether the piece is played in higher lower octave.
4. Mean Velocity (MV) / Variation of Velocity (VV): The velocity of each note indicates how hard a key is pressed in a musical performance, and serves as heuristic for overall volume.
5. Mean Duration (MD) / Variation of Duration (VD): The duration describes for how long each note is pressed in performance, representing articulation, dynamics, and phrasing.
4.1 OVERLAPPING AREA (OA) METRIC
To best capture the salient features within the periodic structure of musical performance, we used a sliding window 2s to construct histograms of the desired feature within each window. We found that representing each performance with such relative measurements better preserved changing dy- namics and stylistic motifs across the entire performance as opposed to a single scalar value (e.g. average note density across the entire performance).
Similar to Yang & Lerch, 2018) Hung et al. (019)), we smoothed the histograms obtained by fitting Gaussian distribution to each feature this allowed to learn compact representation while still capturing the feature's 's variability through its mean (4 and variance o². Then to compare two performances, we computed the Overlapping Area (OA) between the Gaussian pdfs of each fea- ture to quantify their similarity, We demonstrate empirically that this metric identifies the relevant characteristics of interest in our generated performances in Section
EXPERIMENTS
Datasets We used both MAESTRO (Hawthoree et al. 2019) and internal datasets (Simon et al. 2019) for the experimental setup. We used the standard 80/10/10 train/validationntest split from MAESTRO v1.0.0, and augmented the dataset by 10x using pitch shifts of no more than a minor third and time stretches of at most 5%. We note that this augmentation is distinct from the noise- injection procedure referenced in Section 3 the data augmentation merely increases the size of the initial dataset, while the perturbation procedure operates only on the input performance signal. The internal dataset did not require any additional augmentation.
Experimental Setup We implemented the model in the Tensor2Tensor framework (Vaswani et al.) 2017). and used the default hyperparameters for training: 0.2 learning rate with 8000 warmup steps, rsqrt decay, 0.2 dropout, and early stopping for GPU training. For TPU training, we AdaFactor with the rsqrt decay and learning rate warmup steps to be 10K. We adopt many of the hyperparameter configurations from (Huang et al. 20195). where we reduce the query and key hidden size to half the hidden size, use 8 hidden layers, use 384 hidden units, and set the maximum relative distance to consider to half the training sequence length for relative global attention. We set the maximum sequence length be 2048 tokens, and filter size of 1024. We provide additional details on the model architectures and hyperparameter configurations in Appendix C.
5.1 LOG-LIKELIHOOD EVALUATION
As expected, we find that the Transformer autoencoder framework with the encoder output bottle- neck outperforms other baselines. In Tables[]andd 2, we see that all conditional model variants out- perform their unconditional counterparts. Interestingly, we find that for the melody & performance model, different methods of combining the embeddings work better for different datasets. For ex- ample, concatenate led to the lowest NLL for the internal dataset, while sum outperformed all other variants for MAESTRO. We report NLL values for both MAESTRO and the internal dataset for the perturbed-ing input model variants Appendix B.
5.2 SIMILARITY EVALUATION
We use the OA metric from Section to evaluate whether using conditioning signal in both the (a) performance autoencoder and (b) melody & performance autoencoder produces samples that are more similar in style to the conditioning inputs from the evaluation set relative other baselines.
4

---

Under review as a conference paper at ICLR 2020
Model variation
MAESTRO
internal
Unconditional model with rel. attention (Huang et al. 2019b)
1.840
1.49
Performance autoencoder with rel. attention (ours)
1.799
1.384
Table 1: Note-wise test NLL on the MAESTRO and internal datasets, with event best- representa- tions of lengths L 2048. We exclude the performance autoencoder baseline (no aggregation) as it memorized the data (NLL = 0). Conditional models outperformed their unconditional counterparts.
Model variation
MAESTRO
internal
Melody-only Transformer with rel. attention (Huang et al. 2019b)
1.786
1.302
Melody & performance autoencoder with rel. attention, sum (ours)
1.706
1.275
Melody & performance autoencoder with rel. attention, concat (ours)
1.713
1.237
Melody & performance autoencoder with rel. attention, tile (ours)
1.709
1.248
Table 2: Note-wise test NLL on the MAESTRO and internal datasets with melody conditioning, with event-based representations of lengths L 2048. We note that sum worked best for MAESTRO, while concat enate outperformed all other baselines for the internal dataset.
First, sample 500 examples from the evaluation set and use them conditioning signals to generate one sample for each input. Then, we compare each conditioning signal to: (1) the generated sample and (2) unconditional sample. We compute the similarity metric as defined in Section 4 pairwise and take the average over 500 examples. As shown in Table 3, we find that the performance autoencoder generates samples that have 48% higher similarity overall to the conditioning input as compared to the unconditional baseline.
MAESTRO
ND
PR
MP
VP
MV
VV
MD
VD
Avg
Performance (ours)
0.651
0.696
0.634 0.435
0.689
0.693
0.732
0.582
0.692
0.67
Unconditional
0.370
0.466
0.485
0.401
0.606
0.385
0.529
0.46
Internal Dataset
Performance (ours)
0.731
0.837
0.784
0.838
0.778
0.835
0.785
0.827
0.80
Unconditional
0.466
0.561
0.556
0.578
0.405
0.590
0.521
0.624
0.54
Table 3: Average overlapping area (OA) similarity metrics comparing performance conditioned models with unconditional models. Unconditional and Melody-only baselines are from (Huang et al.) 2019b). The metrics are described in detail in Section g The samples in this quantitative comparison are used for the listener study shown the left graph of Figure 4
For the melody & performance autoencoder, we sample 717*2 distinct performances one set 717 for conditioning performance styles, and the other set 717 we use to extract melodies in order to synthesize in the style of different performance. We compare the melody & performance autoencoder to different baselines: (1) one that conditioned only on the melody (Melody-oly)) (2) conditioned only on performance (Performance-onlyy; and (3) unconditional language model. Interestingly, we find that the OA metric is more sensitive to the performance style than melodic similarity. Table 4 demonstrates that the Melody-only autoencoder suffers without the performance conditioning, while the Performance-only model performs best. The Melody & performance autoencoder performs comparably the best model.
reserve
5.3 INTERPOLATIONS
5.3.1
PERFORMANCE AUTOENCODER
In this experiment, test whether the performance autoencoder successfully interpolate be- tween different performances. First, we sample 1000 performances from the internal test set (100 for MAESTRO, due to its smaller size), and split this dataset in half. The first half reserve for
5

---

Under review as a conference paper at ICLR 2020
MAESTRO
ND
PR
MP
VP
MV
VV
MD
VD
Avg
Melody perf. (ours)
0.650
0.696
0.634
0.689
0.692
0.732
0.582
0.692
0.67
Perf-only (ours)
0.600
0.695
0.657
0.721
0.664
0.740
0.527
0.648
0.66
Melody-only
0.609
0.693
0.640
0.693
0.582
0.711
0.569
0.636
0.64
Unconditional
0.376
0.461
0.423
0.480
0.384
0.588
0.347
0.520
0.48
Internal Dataset
Melody perf (ours)
0.646
0.708
0.610
0.717
0.590
0.706
0.658
0.743
0.67
Perf-only (ours)
0.624
0.646
0.624
0.638
0,422
0.595
0.601
0.702
0.61
Melody-only
0.575
0.707
0.662
0.718
0.583
0.702
0.634
0.707
0.66
Unconditional
0.476
0.580
0.541
0.594
0.400
0.585
0.522
0.623
0.54
Table 4: Average overlapping area (OA) similarity metrics comparing models with different condi- tioning. Unconditional and Melody-only baselines are from (Huang al. 2019b). The metries described in detail in Section The samples in this quantitative comparison are used for the listener study shown the right graph of Figure
the original starting performance, which we call "performance and the other half we reserve for the end performance, denoted as "performance B." Then we use the performance encoder to encode performance into its compressed representation 2,, and do the same for performance B obtain zn. For range E [0, 0.125..... .... 0.875, 1.0]. we sample new performance perf that results from decoding o za + (1 - ar) zn- We observe how the OA (averaged across features) defined in Section 4 changes between this newly interpolated performance perf and performances (A, B}.
Interpoletion Performence
0.65
Relatve Clcance
0.50
0.55
0.50
0.45
0.40
0.35
DestPart
0.30
0.75
0,875
10
Figure 2: For the internal dataset, the relative distance from performance A (a 1) to the interpo- lated sample increases is slowly increased to 1.0.
Specifically, we compute the similarity metric between each input performance A and interpolated sample perf for all 500 samples, and compute the same pairwise similarity for each performance B. We then compute the normalized distance between each interpolated sample and the correspond- ing performance A or B, which we denote as: rel.distance (perf A)
Ca_A DA_A D
where the OA is averaged across all features. We average this distance across all elements in the set and find in Figure 2 that the relative distance between performance A slowly increases as we increase from 0 study with oon-aggrega baselines, as we cannot interpolate across variable-length embeddings. We find that similar trend holds for MAESTRO as in Figure 3(a).
expected. We note that it is possible conduct this interpolation
5.3.2 MELODY & PERFORMANCE AUTOENCODER
We conduct a similar study as above with the melody & performance autoencoder. We hold out 716 unique melody-performant performance pairs (melody is not derived from the performance) from the in- ternal evaluation dataset and 50 examples from MAESTRO. We then interpolate across the different performances, while keeping the conditioning melody input the same across the interpolations.
As shown Figure 3(a), we find that a similar trend holds as the performance autoencoder: the newly-interpolatte samples show that the relative distance between performance A increases increase the corresponding value of a. We note that the interpolation effect is slightly lower than that of the previous section, particularly because the interpolated sample is also dependent the

---

Under review as a conference paper at ICLR 2020
0.70
Relative Distance from Interpolation Perfor marormance
07-
Deladive Distanes from Interpolation Melady
8 0.60
Rekatvve Disuaree
0.50
0.45
04
0.8
DaspPrrt
0.
0.05
0.2
03275
0.5
0,22
0.79
0.575
10
00
0.23
028
0.375
0.5
0625
0.75
cam
10
appb
(a) Relative distance to performance A
(b) Relative distance melody
Figure 3: For the internal dataset, relative distance from performance A (a = 1) as is slowly increased to 1.0 while the conditioned melody remains fixed. As (b), we note that the relative distance to the fixed conditioning melody with respect to random performance remains fixed while the interpolation conducted between performances and B, which shows that we can control for elements of style and melody separately.
melody that it is conditioned on. Interestingly, in Figure 3(b), we note that the relative distance between the input performance from which we derived the original melody remains fairly constant across the interpolation procedure. This suggests that are able to factorize out the two sources of variation and that varying the axis of the input performance keeps the variation in melody constant.
5.4 HUMAN EVALUATION
We also conducted listening studies to evaluate the perceived effect of performance and melody conditioning the generated output. Using models trained the internal dataset, conducted two studies: one for performance conditioning, and one for melody and performance conditioning.
Performance Conditioning Study
Metocly Performance Conditioning
Ground Truth
Millody & Performance
Melocly only
Conditored
Per
Uncondit oned
Uncond
60
100
160
200
250
60
100
150
200
250
Number of wins
Number of wine
Figure 4: Results of our listening studies, showing the number of times each source won a pairwise comparison. Black error bars indicate estimated standard deviation of means.
For performance conditioning, presented participants with 20s performance clip from the eval- uation dataset that we used as a conditioning signal. We then asked them to listen to two additional 20s performance clips and conditioning signal. The sources the participants rated included "Ground Truth" (a different snippet of the same sample used for the conditioning signal), "Conditioned" (output of the Performance Au- toencoder), and "Unconditioned" (output of unconditional model). For this study, 492 ratings were collected, with each source involved in 328 pair- wise comparisons.
Likert scale to rate which one sounded most similar in style the
For melody and performance conditioning, we presented participants with a 20s performance clip from the evaluation dataset and a 20s melody from different piece in the evaluation dataset that we used as our conditioning signals. We then asked them to listen to two additional 20s performance clips and use a Likert scale to rate which sounded most like the conditioning melody played in the style of the conditioning performance. The sources the participants rated included "Melody & Performance" (output the
Autoencoder). "Melody only" (output of a model conditioned only the melody signal). "Performance only" (output of model conditioned only the performance signal), and "Unconditioned" (output an unconditional model). For this study, collected, with each source involved in 357 pair-wise comparisons.
7 714 ratings
Figure shows the number of comparisons in which each source was selected as being most similar in style to the conditioning signal. A Kruskal- Wallis H test of the ratings showed that there is least

---

Under review as a conference paper at ICLR 2020
one statistically significant difference between the models: x² = 332.09, p < 0.05 (7.72c-73) for melody conditioning and x2) = 277.74, p < 0.05 (6.53e-60) for melody and performance conditioning, A post-hoc analysis using the Wilcoxon signed-rank test with Bonferroni correction showed that there were statistically significant differences between all pairs of the performance study with p < the "Melody only" and "Melody & Performance" models (p = 0.0894).
0.05/3 / and all pairs of the performance and melody study with p < 0.05/6 except between
These results demonstrate that the performance conditioning signal has clear effect on the gener- ated output. In fact, the effect was sufficiently robust that in the 164 comparisons between "Ground Truth" and "Conditioned".. participants said they had a preference for "Conditioned" 58 times.
Although the results between "Melody-only" and "Melody & Performance" are close, this study demonstrates that conditioning with both melody and performance outperforms conditioning on per- formance alone, and they are competitive with melody-only conditioning, despite the model having to deal with the complexity of incorporating both conditioning signals. In fact, we find quantitative evidence that human evaluation is more sensitive to melodic similarity, as the "Performance-only" model performs worst a slight contrast to the results from the OA metric in Section 5.2.
We provide several audio examples demonstrating the effectiveness of these conditioning signals in htpp///1////// Our qualitative findings from the audio
the online supplement at hhtt:: examples and interpolations. coupled with the quantitative results from the similarity metric and the listening test which capture different aspects of the synthesized performance, support the finding that the Melody & Performance autoencoder offers significant control over the generated samples.
6 RELATED WORK
Sequential autoencoders: Building on the wealth of autoencoding literature (Hinton & Salakhutdi-
nov 2006, Salakhutdinov & Hinton 2009) Vincent et al. 20TO), our work bridges the gap between the traditional sequence-to-sequenec framework (Sutskever et al. (2014), their recent advances with
various attention mechanisms (Vaswani et al. 2017: Shaw et al. 2018 Huang et al., 2019b). and sequential autoencoders. Though (Wang & Wan (019) propose Transformer-based conditional VAE for story generation, the self-attention mechanism is shared between the encoder and decoder. Most similar to our work is that of (Kaiser & Bengio 2018). which uses Transformer decoder and a discrete autoencoding function to map an input sequence into a discretized, compressed represen- tation. We note that this approach is complementary to ours, where similar idea of discretization may be applied to the output of our Transformer encoder. The Music (Roberts et al., 2018) is a sequential VAE with a hierarchical recurrent decoder, which learns an interpretable latent code for musical sequences that can be used during generation time. This work builds upon (Bowman et al. 2015) that uses recurrence and an autoregressive decoder for text generation. Our Transformer autoencoder can be seen as a deterministic variant of the Music with a complex self self-attention mechanism based on relative positioning in both the encoder and decoder to capture more expressive features of the data at both the local and global scale.
Controllable generations using representation learning: There also a wide range of recent work controllable generations, where we focus the music domain. (Engel et al. 2017a) pro- poses to constrain the latent space of unconditional generative models to sample with respect to some predefined attributes, whereas we explicitly define our conditioning signal in the data space and learn a global representation of its style during training. The Universal Music Translation network aims to translate music across various styles, but is not directly comparable with raw audio waveforms (Mor et al.) 2018). Both (Meade et al. 2019) and MuseNet (Payne 2019) generate music based on user preferences, but adopt slightly different approach: the models specifically trained with labeled tokens (c.g.. composer and instrumentation) as conditioning input, while our Transformer autoencoder's global style representation is learned in an unsupervised way.
approach they work
7 CONCLUSION
We proposed our Transformer autoencoder for conditional music generation, a sequential autoen- coder model which utilizes an autoregressive Transformer encoder and decoder for improved mod- eling of musical sequences with long-term structure. We show that this model allows users to easily
8

---

Under review as a conference paper at ICLR 2020
adapt the outputs of their generative model using even single imput performance. Through experi- ments on the MAESTRO and internal datasets, we demonstrate both quantitatively and qualitatively that model generates samples that sound similar in style to variety of conditioning signals relative to baselines. For future work, it would be interesting to explore other training procedures such variational techniques few-shot learning approaches account for situations in which the input signals are from slightly different data distributions than the training set.
REFERENCES
Pierre Baldi. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML. workshop unsupervised and transfer learning, pp. 37 37-49, 2012.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Ben- gio. Generating sentences from a continuous space. arXiv preprint drXiv:1571.06349, 2015.
Jesse Engel, Matthew Hoffman, and Adam Roberts. Latent constraints: Learning to generate condi- tionally from unconditional generative models. arXiv preprint
2017a.
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Mohammad Norouzi, Douglas Eck, and Karen Simonyan. Neural audio synthesis of musical notes with wavenet autoencoders. In Proceedings of the 34th International Conference Machine Learning-Volle Volume 70, pp. 1068- 1077. JMLR. org, 2017b.
Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, and Adam Roberts. Gansynth: Adversarial neural audio synthesis. arXiv preprint 1rXiv:1902.08710, 2019.
Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, and David Bamman. Learning to groove with inverse sequence transformations. arXiv preprint arXiv: 0w01//90000611 08,, 2019.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
David Ha and Douglas Eck. arXiv:/704.033 2017.
neural representation of sketch drawings.
arXiv preprint
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel. and Douglas Eck. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Represen- fations, 2019. URL. http::ttts///t..tt bittpps://pperevvzi eww net/
Geoffrey E Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(578)) 507, 2006.
Cheng- Zhi Anna Huang, Tim Cooijmans, Adam Roberts, Aaron Courville, and Douglas Eck. Coun- terpoint by convolution. arXiv preprint wrXiv:1903.07227. 2019a.
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, and Douglas Eck. Music transformer: Generating music with long-term structure. In International Conference on Learning Representations, 2019b. URL https: http://pppreviiieeee
Hsiao-Tzu Hung, Chung- Yang Wang, Yi-Hsuan Yang, and Hsin-Min Wang. Improving automatic jazz melody generation by transfer learning techniques. arXiv preprint arXiv: 09484494444 2019.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
Eukasz Kaiser and Samy Bengio. Discrete autoencoders for sequence models. arXiv preprint 100.997801 2018.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint 1312 2013.

---

Under review as a conference paper at ICLR 2020
Nicholas Meade, Nicholas Barreyre, Scott C Lowe, and Sageev Oore. Exploring conditioning for generative music systems with iuman-itterpretable controls. arXiv preprint arXiv:1907.043:0 2019.
Noam Mor, Lior Wolf, Adam Polyak, and Yaniv Taigman. A universal music translation network. arXiv preprint 1e00.100550888 2018.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for 2016.
raw audio. arXiv preprint
Sageev Oore, Ian Simon, Sander Dieleman, Douglas Eck, and Karen Simonyan. This time with feeling: learning expressive musical performance. Neural Computing and Applications, pp. 1- 13, 2018
Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. Image transformer. arXiv preprint arXiv:/800.005751, 2018.
Christine Payne. Musenet, 2019. URL
Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck. A hierarchical latent vector model for learning long-term structure in music. arXiv preprint arXiv: 1803.00800 2018.
Ruslan Salakhutdinov and Geoffrey Hinton. Deep boltzmann machines. In Artificial intelligence and statistics, pp. 448-455, 2009.
Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions. arXiv preprint arXiv:1803.00 000.555 2018.
Ian Simon, Cheng-Zhi Anna Huang, Jesse Engel, Curtis Hawthorne, and Monica Dinculescu. Gen- erating piano music with transformer. 2019. htttpps:/mmaeettteeensott
gcano-transforre
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, pp- 3483-3491, 2015.
Casper Kaae Sønderby, Tapani Raiko, Lars Maalee, Søren Kaae Sønderby, and Ole Winther. Ladder variational autoencoders. Advances in neural information processing systems, pp. 2016.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances neural information processing systems, pp. 3104-3112, 2014.
Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Con- ditional image generation with pixelenn decoders. In Advances in neural information processing systems, pp. 4790-4798, 2016.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096-1103. ACM, 2008.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with local denoising criterion. Journal of machine learning research, ((Dec):3371-3408 2010.
Tianming Wang and Xiaojun Wan. T-evae: Transformer-basee conditioned variational autoencoder for story completion. In Proceedings of the Twenty-Eige Eighth International Joins Conference on Artificial Intelligence, IJCAI-19, pp. 5233-5239 International Joint Conferences on Artificial Intelligence Organization, 7 2019. doi: (0.24963/ijcai.2099/77 URL https: :/ttps://dd. //tp:////d//////////////o 10. 24963/463392 cai 201 9/77
10

---

Under review as a conference paper at ICLR 2020
Li-Chia Yang and Alexander Lerch. On the evaluation of generative models in music. Neural Computing and Applications. pp. 1-12, 2018.

---

Under review as a conference paper at ICLR 2020
APPENDIX
FURTHER DETAILS ON INPUT PERTURBATION PROCEDURE
We elaborate upon the perturbation procedure as follows, which only applies the melody & per- formance conditioning music generation task. First, we take a target performance that we would like our model to predict and extract the corresponding melody from this performance. We use this "clean" melody as part of our conditioning signal. Then, we modify the conditioning performance by either shifting the pitch up or down 6 semitones and stretching the timing by + 5%. Then for each new data point during training, single noise injection procedure is randomly sampled from the cross product of all possible combinations of pitch shift values and time stretch values (evaluated in intervals of 2.5%). At test time, the data points are left unperturbed.
NLL EVALUATION FOR "NoISY" MODEL
Below, we provide the note-wise test NLL on the MAESTRO and internal datasets with melody con- ditioning, where the conditioning performance is perturbed by the procedure outlined in Section
Model variation
MAESTRO
internal Dataset
Noisy Melody TF autoencoder with relative attention, sum Noisy Melody TF autoencoder with relative attention, concat Noisy Melody TF autoencoder with relative attention, tile
1.721
1.248
1.719
1.249
1.728
1.253
Table 5: Note-wise test NLL on the MAESTRO and internal piano performance datasets with melody conditioning, with event-based representations of lengths L 2048.
C ADDITIONAL DETAILS ON MODEL TRAINING PROCEDURE
We emphasize that the Transformer trained an autoencoder-like fashion. Specifically, for performance-olly conditioning, the Transforemr decoder is tasked with predicting the same perfor- mance that was fed as input to the encoder. this way, we encourage the model to learn global rep- resentations (the mean-aggregated performance embedding from the encoder) that will faithfully be able to reconstruct the input performance. For melody performance conditioning, the Transformer autoencoder is trained to predict new performance using the combined melody-peeffrmancc bedding, where the loss is computed with respect to the conditioned input performance that is pro- vided the encoder.
D MODEL ARCHITECTURE AND HYPERPARA METER CONFIGURATIONS
We mostly use the default Transformer architecture as provided in the Tensor2Tensor framework, such self-attention heads listed in the main text, and list the slight adjustments we made for each dataset below:
D.1 MAESTRO
For the MAESTRO dataset, we follow the hyperparameter setup of (Huang et al. 2019b
1. num hidden layers
2. hidden units 384
3. filter size 1024
4. maximum sequence length 2048
5. maximum relative distance half the hidden size
6. dropout = 0.1
12

---

Under review as a conference paper at ICLR 2020
D.2 INTERNAL DATASET
For the internal dataset, we modify the number of hidden layers and slightly increase the level of dropout.
num hidden layers 8
2. hidden units 384
3. filter size 1024
4. maximum sequence length 2048
5. maximum relative distance half the hidden size
6. dropout = 0.15
INTERPOLATIONS DETAILS
Interpolation relative distance results for the (a) performance and (b) melody & performance Trans- former autoencoders for the MAESTRO dataset.
0.70
Relattve Oistance from Interpoiction Performance
Rebiive Distance from Interpolation Perforaace A
0.60
0.60
Relallve Disaavee
0.4
0.45
0.40
0.5
Das|Port
Daathar
0.
0:95
025
0375
05
0,25
0.75
0.875
10
0.0
0.23
022
0.75
05
0.655
0.7
0875
10
apha
(a) Relative distance from interpolated sample the Relative distance from the interpolated sample original starting performance.
original melody, which kept fixed.
Figure 5: The distance the original performance increases as the value of increases in (a), as expected. In (b), we see that there is a very slight increase the relative distance to the original melody during the interpolation procedure.
13

---

Under review as a conference paper at ICLR 2020
F INTERNAL DATASET PERFORMANCE INTERPOLATIONS
Here, we provide piano rolls demonstrating the effects of latent-space interpolation for the internal dataset, for both the (a) performance and (b) melody & performance Transformer autoencoder. For similar results in MAESTRO as well as additional listening samples, we refer the reader to the online supplement: http:
(a) Original starting performance
(b) CE 0.125
(c) a 0.375
(d) a 0.5
Gaavel
(002)
5hs
(c) a 0.625
(f) = 0.875
20000 MANDO
32
(D0c)
(g) 1.0 (reconstruction)
(h) Original final performance
Figure 6: Interpolation of a starting performance (a) from the internal dataset to a final performance (h), with the coefficient a controlling the level of interpolation between the latent encodings between the two performances.
14

---

Under review as a conference paper at ICLR 2020
G
INTERNAL DATASET MELODY INTERPOLATION
Dioe (loCc)
Arre (1aN)
(a) Original melody
(b) Original starting performance
20000 MAND11
32
tire (b00)
Shs (ten
(c) 0 (reconstruction)
(d) 0.125
Bin (00C)
Shs (ded)
(e) 0.375
(f) 0.5
DANN MANDN
BiTo (D02)
Sns (Bod)
(g) 0.625
(h) 0.875
p100 MANDN
12.
gate MADD S B
DV10 (002)
205 (500)
(i) 1.0 (reconstruction)
(j) Original final performance
Figure 7: Interpolation of a starting performance (b) from the internal dataset to a final performance (i), with the coefficient controlling the level of interpolation between the latent encodings between the two performances. The original conditioning melody (a) is kept fixed throughout the interpola- tion.
15


Reviews:
[{"review_id": "rJlAcEsTtS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## summary\nIn this paper, the author extends the standard music Transformer into a conditional version: two encoders are evolved, one for encoding the performance and the other is used for encoding the melody. The output representation has to be similar to the input. The authors conduct experiments on the MAESTRO dataset and an internal, 10,000+ hour dataset of piano performances to verify the proposed algorithm.\n\n## Novelty \nThe application is interesting, but the novelty of the architecture itself is limited. Multiple encoder structure has been widely investigated in machine translation.\n\n## Questions\n1.\tIn section 4.2, how do you use the $\\mathcal{Y}$? Since it is defined but never used. What does the $p()$ and $q()$ mean ? You mentioned that “We omit the usual first term in the MMD loss …” but if so, why do you introduce this term to evaluation metric?\n2.\tBy checking the music Transformer, in Table 3, it is not surprising to see that the proposed method outperforms the corresponding baselines, because no conditional information is used. \n3.     It is better to give some mathematical definition of music generation with specific style. I am not working on music generation but I list two CV related papers about conditional image translation, which mathematically describes \"an image with specific style\".\n4.\tConsidering that this is an unsupervised setting that two styles are transformed, can cycle-consistency be implemented as a baseline? The following two papers are about conditional unsupervised image-to-image translation, which build a cycle-consistency loss during the feedback and might help improve the performances.\n\n\n## Reference\n[ref1] Multimodal Unsupervised Image-to-Image Translation, ECCV’18\n[ref2] Conditional image-to-image translation, CVPR’18\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Hygtfbj6FH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a technique for encoding the high level “style” of pieces of symbolic music. The music is represented as a variant of the MIDI format. The main strategy is to condition a Music Transformer architecture on this global “style embedding”.  Additionally, the Music Transformer model is also conditioned on a combination of both “style” and “melody” embeddings to try and generate music “similar” to the conditioning melody but in the style of the performance embedding. \n\nOverall, I think the paper presents an interesting application and parts of it are well written, however I have concerns with the technical presentation in parts of the paper and some of the methodology. Firstly, I think the algorithmic novelty in the paper is fairly limited. The performance conditioning vector is generated by an additional encoding transformer, compared to the Music Transformer paper (Huang et. al. 2019b). However, the limited algorithmic novelty is not the main concern. The authors also mention an internal dataset of music audio and transcriptions, which can be a major contribution to the music information retrieval (MIR) community. However it is not clear if this dataset will be publicly released or is only for internal experiments. \n\nIn terms of technical presentation, I think the authors should clarify how the model is trained. It took me a couple of passes and reading the Music Transformer paper to realise that in the melody and performance conditioning case, the aim is to generate the full score (melody and accompaniment) while conditioning on the performance style and melody (which is represented using a different vocabulary). This point can be easily clarified in Figure 1, by adding the input to the encoder as input to the decoder for computing the loss. Although I understand the need for anonymity and constraints while referring to unreleased datasets, it would still be useful for the reader/reviewer to have some details of how the melody was extracted and represented. “An internal procedure” is quite mysterious. \n\nMeasuring music similarity is a difficult problem and the topic has been the subject of at least 2 decades of research. I find the description of the “performance feature” to be lacking in necessary background and detail. Firstly, I am not sure what the final dimensionality of the feature vector is. Is it real valued? The authors mention (Yang and Lerch, 2018) but use a totally different set of attributes compared to that paper. I also don’t see the connection between this proposed feature vector and using the IMQ kernel for measuring similarity. This connection is not motivated adequately and after reading (Jitkrittum et. al. 2019) its not obvious to me why this is the most appropriate metric. Finally, it would be useful if the authors comment on existing methods for measuring music similarity in symbolic music and how their proposed feature fits into existing work. A lot of work has been published on this topic, most recently in the context of Query-by-Humming [1]. \n\nMinor Comments\n\n1. “...which typically incorporate global conditioning as part pf the training procedure” Could you elaborate on this point? Is the global conditioning the samples from the noise distribution? \n2. Figure 1 should be clarified or another figure should be added to show how the melody conditioning works. Maybe a comment on the melody vocabulary or a reference would also be useful. \n3. The MAESTRO dataset is described in terms of the number of performances while the internal dataset is described in terms of the number of hours of audio. Its not possible for the reader to get a sense of the relative sizes of the 2 datasets and how the results should be interpreted. \n4. There should be more background and description in Section 4. Where does the performance feature come from? Why use this feature compared to existing techniques for measuring similarity between symbolic music pieces? Is it computational efficiency? Why not compare the conditioning melody with the generated performance similar to query-by-humming? Where does the IMQ kernel come from? What is the size of the feature vector? \n5. In section 5.2, a conditioning sample, a generated sequence and an unconditional sample are used to compute the similarity measure. Which terms do these correspond to in the MMD-like term (x,y,y’)? \n6. I like the experiments performed in Section 5.3 with the linear combination of 2 performance embeddings. \n\n[1] A Survey of Query-By-Humming Similarity Methods: http://vlm1.uta.edu/~athitsos/publications/kotsifakos_petra2012.pdf\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Main content:

Blind review #3 summarizes it well:

This paper presents a technique for encoding the high level “style” of pieces of symbolic music. The music is represented as a variant of the MIDI format. The main strategy is to condition a Music Transformer architecture on this global “style embedding”.  Additionally, the Music Transformer model is also conditioned on a combination of both “style” and “melody” embeddings to try and generate music “similar” to the conditioning melody but in the style of the performance embedding. 

--

Discussion:

The reviewers questioned the novelty. Blind review #2 wrote: "Overall, I think the paper presents an interesting application and parts of it are well written, however I have concerns with the technical presentation in parts of the paper and some of the methodology. Firstly, I think the algorithmic novelty in the paper is fairly limited. The performance conditioning vector is generated by an additional encoding transformer, compared to the Music Transformer paper (Huang et. al. 2019b). However, the limited algorithmic novelty is not the main concern. The authors also mention an internal dataset of music audio and transcriptions, which can be a major contribution to the music information retrieval (MIR) community. However it is not clear if this dataset will be publicly released or is only for internal experiments."

However, after revision, the same reviewer has upgraded the review to a weak accept, as the authors wrote "We emphasize that our goal is to provide users with more fine-grained control over the outputs generated by a seq2seq language model. Despite its simplicity, our method is able to learn a global representation of style for a Transformer, which to the best of our knowledge is a novel contribution for music generation. Additionally, we can synthesize an arbitrary melody into the style of another performance, and we demonstrate the effectiveness of our results both quantitatively (metrics) and qualitatively (interpolations, samples, and user listening studies)."

--

Recommendation and justification:

This paper is borderline for the reasons above, and due to the large number of strong papers, is not accepted at this time. As one comment, this work might actually be more suitable for a more specialized conference like ISMIR, as its novel contribution is more to music applications than to fundamental machine learning approaches.
Decision: Reject