Composing Features: Compositional Model Augmentation for Steerability of Music Transformers
Authors: Halley Young
Vincent Dumoulin
Pablo Samuel Castro
Jesse Engel
Cheng-Zhi Anna Huang
Venue: ICLR 2022 Submitted (2022)
Keywords: applications
music
controllable generation
compositionality
transformer
finetuning

Abstract: Music is a combinatorial art. Given a starting sequence, many continuations are possible, yet often only one is written down. With generative models, we can explore many. However, finding a continuation with specific combinations of features (such as rising pitches, with block chords played in syncopated rhythm) can take many trials.
To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight fine-tuning methods such as prefix tuning and bias tuning. We introduce a novel contrastive loss function that enables us to steer compositional models over logical features using supervised learning. We examine the difficulty in steering based on whether features musically follow a prime or not, using existing music as a proxy. We show that with a relatively small number of extra parameters, our method allows bias tuning to perform successful fine-tuning in both the single-feature and compositional setting.

Under review as a conference paper at ICLR 2022
COMPOSING FEATURES: COMPOSITIONAL MODEL AUGMENTATION FOR STEERABILITY OF MUSIC TRANSFORMERS
Anonymous authors Paper under double-bling review
ABSTRACT
Music is a combinatorial art. Given starting sequence, many continuations are possible, yet often only one written down. With generative models, we can explore many. However, finding a continuation with specific combinations of fea- tures (such as rising pitches, with block chords played in syncopated rhythm) take many trials. To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight fine-tuning methods such prefix tuning and bias tuning. We in- troduce novel contrastive loss function that enables us to steer compositional models over logical features using supervised learning. We examine the difficulty in steering based on whether features musically follow a prime or not, using ex- isting music as a proxy. We show that with relatively small number of extra parameters, our method allows bias tuning to perform successful fine-tuning in both the single-feature and compositional setting.
INTRODUCTION
Recent research has focused on ways of adapting unconditional language models to perform well on conditional tasks they were not initially trained on. Methods such fine-tuning, side-tuning. bias-tuning, and prompt or prefix tuning have emerged as lead candidates for such tasks such as steering the sentiment of or words mentioned in sentence (Ben-Zaken et al., 202 Li & Liang, 2021 Lester et al.) 2021] Dathathri et al., (020). However, such tasks are normally presented in a non- or minimally wocollyootitionnnt approach (possibly controlling for two orthogonal variables, such as sentiment and topic (Keskar et al.) 2019). but rarely more than that). In contrast, in the domain music generation, sequence-conee contincatcoiiitioonn is inherently highly compositional problem: the user likely has many aspects of the output they would like to control, such as speed, dynamics, harmony, or texture, each of which can be decomposed into multiple sub-features. Furthermore, the relationship between these features and the output is more diffuse than in the NLP settings: while in examples such as Keskar et al.] (2019), individual words indicate the different conditions being satisfied, in music the entire sequence of tokens instance, average pitch over span of time depends on all tokens representing that span). This makes compositionality even more challenging than in the text generation setting. On the other hand, music is a sequential domain where there are clear logical features which can be examined, such average dynamics number of notes per second. This further motivates using music test-bet bed for highly compositional tasks involving simultaneously steering an autoregressive model towards several particular desired attributes according to different classifiers.
used in evaluating single feature (for
As more immediate motivation, consider the following scenario: composer wants to sample from the pre-trained Music Transformer (Huang et al., 2018) to complete a musical phrase. addition to the overall musical quality of the continuation, they want it to stay in key and switch to using block chords (i.e. a few notes played together at once). Repeatedly sampling continuations from the model and cherry-picking good sample (rejection sampling) would be very labor-intensive, but assuming they can compute binary features for "stays in key" and "uses block chords", the composer could sample a large number of continuations and cherry-pic from the smaller set of continuations which exhibit all features (thereby delegating part of the accept/reject step). While significantly less labor-intensive, this solution could potentially require sampling enormous amounts of continuations

---

Under review as a conference paper at ICLR 2022
Original continuation
Prime
"Uses block chords"
Φj
Steerable Transformer
Many block chords!
Only one
block chord
Figure 1: Showcasing prime from the test set steered in two different ways. The original contin- uation has arpeggiation (the steep rising and falling lines) throughout. The right piano rolls were steered with logical feature that checks for many simultaneous notes (block chords). In particular, the bottom right piano roll shows that the model learned to "exploit" the logical feature by repeating low note, while still sounding musical.
if, for instance, the requested features significantly differ from those the priming sequence (such that it and the desired continuation form unlikely sequence according to the generative model). In other words, the pre-trained model could be a poor proposal distribution for some applications.
In this work, are interested in developing better proposal distributions through adaptation ap- proach which can steer the generative model towards continuations which i) are significantly more likely to exhibit the requested feature, and ii) exhibit satisfactory musical quality. The approach should be able to accommodate a large number of features without adding significant memory or computation overhead. We achieve this by making features composable, making it possible steer features independently and also multiple features at once. Figure I shows that when using pre- trained transformer model augmented with relatively small number of additional parameters, are able to steer towards arbitrary logical music features and achieve realistic music generation sim- ply by sampling directly from the model ! In contrast, the same unconditional transformer model fails to produce any examples satisfying those features even when generating 100 samples. With our method, a composer can control both the short-chunk features, and long-form structure, by chain- ing together chunks steered in different directions (using different combinations of features), while maintaining long-term coherence (by leveraging the transformers' full self-attention receptive field). See the demo in the footnote for longer steered examples, and compositions cemi-automatically gen- erated using our feature tuning approach a -isiciannddiected way.
PROBLEM FORMULATION
Music Transformer is an autoregressive language model which decomposes the joint probability of sequence of tokens 21,..., (where In K., and K. is set of categorical tokens) into
p(x = 21, IN) = p(x1) Il
p(an L1 FI....,,--I)
In-
(1)
7=
leverages common modeling approach which represents the conditional probabilities p(u,, Ti, ... ,.....,,))))) using neural network (Bengio al. 2003). As its implies, Music Transformer uses Transformer network architecture (Vaswani et al. 2017a). Each token 2,, is first mapped to real-valued embedding e, (for instance using a lookup table), then the network maps each sequence ej...., - probability distribution for the value of over the elements of K.
Sequence continuation an autoregressive language model works by repeatedly sampling from its distribution over the next token given the previous tokens. Starting from some priming se-
Listen these examples at https:://sttrageeeeesee eapis. wwt.cco..cccoccccc ng- features/
index. html
2

---

Under review as a conference paper at ICLR 2022
quence Xp (Zo,.... .... FMM) first sample ya++1 p(- (o....;MM) then VM++2 p(- To. .......... 2M, ........M.
MM-1), and so until the end of the continued sequence X. (yM+1..... yN Many downstream tasks can be cast as sequence continuation problems, including the steerable music generation problem investigated in this work.
{ E K - {0, 1]. Each @ takes the value if prime-continuatio pair exhibits that feature (which we note as (xp, } oj), and 0 otherwise. Note that the features must take both prime and continuation sequences input, since some contin- uation features may be relative to the priming sequence (e.g. "significantly higher pitch"). Our true objective with respect feature Ø steer the model towards distribution which maximizes
We are given a set of features
Ex. xp [(xp; Xx) - Φ5]
(2)
while maintaining musicality. This objective is non-differentiable because (Xp, x.) - is non- differentiable satisfiability criterion.
In addition to the single-feature feature problem, we also consider the problem of composed features é, i.c.
(Xp: Xx) F Φ
(xp: Xx) } 6j. PGEáÇO
account for scenarios where a user is interested in steering the model towards multiple features (such as in the "stays in key" and "exhibits extreme dynamical contrast" scenario dicussed in the introduction).
3 PROPOSED APPROACH
We start by describing our proposed approach in the single-fcature case and later on explain how we adapt it to the compositional case.
3.1 LIKELIHOOD-BAND BASED TRAINING
While approaches using reinforcement learning such as KL-regularized deep Q-learning QJques al., 2016)-could be used to overcome the non-differentiabiliiy problem, in this work we consider a proxy loss in the form of the negative log-likelihood
= log (x Xp);
(4)
which we use in two ways:
1. Positively: given prime-continuation pair (Xp, X.) } Φj. we find adaptation 0, of the model's parameters that minimizes /, (we use the symbol v to denote the fact that / is computed using the correct parameters 0,). By using criticcontinuation examples that sound musical, we ensure that the steered model stays musically grounded.
2. Negatively: we can also take advantage of other features Φi for which (Xp: x.) H di by maximizing lx (we use the symbol x denote the fact that [ computed using the incorrect parameters 0z).
The positive case corresponds maximum-likeliben training. Additionally, we can exploit the intuition that the adapted parameters 0, should "explain" the prime-continuation pair (Xp, x.) } Φj better than 6, (for some feature d; for which (Xp. x.) I 6.) & (the non con-adapted model parameters, with a corresponding loss lg). In other words, we can maximize the probability of choosing f, over 0 and 0 by minimizing a contrastive loss of the form
To simplify the discussion, we assume fixed sequence length N, but the explanation applies to sequences of varying lengths well.
3

---

Under review as a conference paper at ICLR 2022
Maestro dataset
-log Pg,
Steerable Transformer
YOi
lx
log Po,
Unconditional Transformer
1g =
log Pa
Prime
Continuation
E elyy lv
=a - a) log
lx + 40
Overall training setup, where the (positive) feature being learned is Φy (with parameters 0z), and the negative feature is de (with parameters 0,). The parameters 0, used for the unconditional negative log-likelihood, are not learned.
Figure 2:
Po, (Xa Xp)
et xp) -log c-lv + texx c
- log
(5)
po, (xe
Xp)
Pa.
(Xe
Xp) + px(
We propose a loss that interpolates between Equations 4 and using an a coefficient (which is treated as a hyperparameter):
elv a. (1 a1- log ( te-tx te
(6)
Intuitively, the maximum-likelihode setting should suffice to achieve our adaptation goals, but in practice find that the approach benefits from the inclusion negative cases through a contrastive loss term. We tried different a values and found = 0.8 to work well in practice. See Figure 2 for an illustration the training setup.
3.2 DEATURE-CONDITIONAL ADAPTATION
Fine-tuning all model parameters can be prohibitive if the number J of features is large (let alone combinatorially large in the compositional case); however, recent work provides effective and lightweight alternatives:
Prefix tuning (Li & Liang
2021 works by preprending learnable task embeddings e- e-1 to the priming sequence embeddings eI.... eM The loss gradient is then backpropagated through the language model and into the task embeddings.
2. Bias-tuning (Ben-Zaken et al. 2021) works by adapting a small subset O C of the transformer's parameters, namely the biases its affine transformations. Since these biases amount to a small fraction of the model's parameters, in the case where the number of tasks is relatively small, tuning separate Ag for each task becomes feasible. We present extension to bias-tuning where the number of tasks is exponential in the number of total classification functions, using approach which nevertheless only requires number of tuned parameters linear in the number of total classification functions.
In practice, while prefix tuning showed promise in the single-feature setting, we were unable to make it work the compositional setting. therefore focus our investigation on bias-tuning and present prefix tuning results in the Appendix.
In the compositional setting, naive approach requires learning 2 2 model adaptations. Instead, we propose to express the adaptation for a composed feature as the combination of the 0, of its underlying features Φ $ More specifically, for bias-tuning average the adapted biases as

---

Under review as a conference paper at ICLR 2022
Embedding layer
Inner layer
Inner layer
Input
Output
Bias vector
Bias vector
Coerat/vvergg
Average
Average
MLF MLP
MLP
WLP
MLF
MLP
01
Prefix tuning
Bias tuning
Emb.
Figure 3: Prefix- and bias-tuning methods used to steer the music transformer.
Σ
Bb.j
6 E
Note that we do not simply use the above heuristic to compose feature adaptations post-hoc; we train the model a compositional setting (by sampling prime-comtinuation pairs (Xp, X.) |= for vari- composed features) that the single-feature adaptations can learn to work well in conjunction with each other. See/Figure for an illustration the methods.
METHODOLOGG
4.1 MUSICAL FEATURES
We consider 18 features representing properties that composer would typically like to control; the Appendix for a complete list. Features include both absolute and relative features. Absolute features apply only to the continuation, while relative features, such "the continuation is signifi- cantly higher in pitch than the prime" or "the continuation significantly more rhythmically dense than the prime" describe relations between the prime and continuation. Note that this type of model should work with any feature function which takes in sequences and returns a boolean; we chose examples that have clear musical meanings and are easy implement.
In addition to exploring binary musical features, we also explore fine-tuning to provided dataset; in our experiments we fine-tune the Maestro dataset (Hawthorne et al. 2019). and refer this feature as óg.
4.2 TRAINING DATA
The prime-continuation pairs used to train the feature-specific adaptations are drawn from the Mae- stro dataset (Hawthorne et al. 2019), open-source collection of virtuoso piano performances. Each prime is 100 tokens long (approximately 2- 10 seconds), and each continuation is also 100 tokens long. Note that since all prime- contincontination pairs are drawn from the Maestro dataset, implicitly assume that (xp, x.) = Φz always holds.
In the single-prefit setting, when a prime-comee contin-cnniiiationn pair (Xp. Xx) is sampled for feature Φj, also select another feature Φ; at random such that (X,, X.) ifr Øs and use its corresponding 8; in Equation 6
In the compositional case, we first extend so that each feature has a corresponding "negated" feature (e.g. "has both loud and soft pitches" "no contrast in dynamics"), bringing the total

---

Under review as a conference paper at ICLR 2022
Non-contrastive
Contrastive
2.75
2.50
Unconditional loss
2.25
MLE Loss
Method
tuning (cofactual)
2.00
Bias tuning (counterfactual)
Dataset
1.75
Train
Validation
1.50
1.25
1.00
7
Training epoch
Training epoch
Figure 4: MLE loss during training of the bias tuning method. Using the contrastive loss is more effective at discriminating between cofactual and counterfactual features (see subsection 5.2).
number of features to 2 | 36. By definition, each Frime-continuation pair exhibits exactly 18 features. When sampling a pair (Xp:X.), we compute / with respect random subset of those 18 features (with size drawn at random from U(1, 12)). We compute Ix by negating some of the sampled features.
4.3 IMPLEMENTATION
In the single-feature case, we use standard prefix tuning model as per Li & Liang (022)) sociating every feature oj with a prefix Vy. As in their paper, we used the finding that rela- tively low-dimensional embedding space of feature prefixes (200 dimensions) projected to higher- dimensional embedding (2048 dimensions) using a shared MIP across feature prefixes worked well. The pre-trained transformer model had parameters modified, but changed accept these 2048-diman -448-dimensional vectors as 4 512-dimensional vectors prepended before the 512-dimensional vec- tors representing the input tokens (as the transformer's latent embedding was 512 dimensions). The resulting sequence of vectors were masked using causal attention in order to predict x..
The loss used in the compositional setting almost identical to Equation 6 the main difference being in the contrastive loss implementation. The features used to compute Ix are obtained by negating some of the features used compute l.. and the fraction 8 of negated features is used to modulate the interpolation coefficient as a' = min(a, 9). The intuition is that the differences steering should be more noticeable when comparing feature sets with less overlap.
REGIME
5.1 TRAINING STATISTICS
As seen in Figure 4 in the contrastive case the model not only learns assign progressively more weight to (xp, Uy) (the "positive" examples), but also learns to assign progressively less weight (Xp.Xe x²(1s) (the "negative" examples).
5.2 THE LARGE VARIANCE DIFFICULTY
PAIRS
As shown in Figure 6 there is large difference in the efficacy of rejection sampling with respect to primes from the Maestro validation dataset which are followed the Maestro validation dataset by satisfying continuation (here termed "cofactual primes"), and rejection sampling with respect to primes from the maestro validation dataset which in fact are not followed by a satisfying contin- uation ("counterfactual primes"). In fact, for some continuations the "counterfactual" probability of

---

Under review as a conference paper at ICLR 2022
Prefix tuning (Contrastive)
Prefix tuning (Non-contrastive)
Unconditional
Cofactual
0.58 (0.57, 0.59)
0.54 (0.53, 0.55)
0.38 (0.37, 0.39)
Counterfactual
0.39 (0.38, 0.40)
0.33 (0.32, 0.34) 0.40 (0.39, 0.41)
0.23 (0.22, 0.24) 0.27 (0.26, 0.27)
Random 0.43 (0.42, 0.44)
Table I: Comparison of the efficacy of Prefix tuning (using contrastive and non-contrastive) with the efficacy of the unconditional approach. The numbers in parentheses indicate 95% confidence intervals.
efficacy is almost 0%. Prefix tuning not only increases the efficacy of steering overall, but also tends to decrease the relative difficulty of "counter-factual" steering.
5.3 CONTRASTIVE LOSS GENERALLY IMPROVES EFFICACY OVER NON-CONTRASTIN L.OSS
While efficacy differs for different primes and features, contrastive learning generally signifi- cantly outperforms non-contrastive learning, which almost universally outperforms the uncondi- tional model. As seen in Table I] it is usually (but not always) the case that the efficacy of the model trained with contrastive loss is significantly higher than that trained with non-contrastive loss, and very rarely do both contrastive and non-contrastive models fail (i.e., fail to perform significantly better, and actually seem to do worse, than the unconditional model).
Difficulty vs. Efficacy
Cofactual
Random
0.8
Condition
0.8
Probability achievement
8.7
Messtre
Uncondkooai
0.8
Ettcacy
24
0.4
23
0.2
Uncondilional
Non- contrastive
Contrasie astive
0.0
Difficulty
Number steered features
Number steered features
Figure 5: Left: Difficulty is quantified as the efficacy of the unconditional model. In most cases, contrastive learning significantly improves the dashed orange lines). In only few cases does prefix tuning fail to improve over the uncon- ditional model. Middle and right: Efficacy of various methods with various numbers of steered features on cofactual and random features.
non-contrastive learning (as evidenced by
6 RESULTS IN THE COMPOSITIONAL SETTING
Steering unconditionally becomes intractable as the number of steered features increases. Under naive assumption of independence among feature presence, one would assume that the rejection sampling efficacy in steering total of n features would roughly equal p", where is the mean probability of feature being satisfied. While this assumption naive, do find that in the unconditional setting, the level of efficacy of unconditional sampling drops substantially as the number of features be steered increases. On average, only 0.2% of the time were cofactually steered features all achieved correctly using rejection sampling.
Trained models degrade at a much slower rate as the number of steered features increases. While the bias-tuning model had average efficacy of 59% in the single prefix setting with randomly sampled prefixes, it dropped by an order of magnitude (to 7.9%) in the randomly-sampled 6-prefix setting (still significantly outperforming unconditional rejection sampling).
Bias-tuning is the most effective way to steer a model as the number of steered features in- creases. This was true both for "cofactual" steered features and random steered features (although
7

---

Under review as a conference paper at ICLR 2022
random steered features understandably was harder task overall). Notably, though prefix tun- ing had performed well when trained on single prefixes, failed to generalize to the compositionally. trained case even in the cases that single prefix was being steered.
Listening test The above experiments evaluate the effectiveness of methods in satisfying the re- quested features. To evaluate the musical quality of these 'successful' generations, conducted a listening study with musicians to compare the steering of three features between our overall best augmentation approach (i.e. bias tuning) and the baseline of rejection sampling on the unconditional model. Questions were posed as pairwise comparisons, and listeners were asked to rate which one they thought was more musical. To prepare the samples, randomly picked 120 primes from the test set. For steering each prime, we randomly chose three-feature sets that were present in the dataset the conditioning features. We asked eight musicians each rate fifteen pairs. The results show that our bias- tuning approach was much preferred over the unconditional, and the results were statistically significant (p 0.0003). Bias tuning won 63 of the pairwise comparisons, tied for 26 pairs, and lost for 31 pairs. This shows that our bias-tuning approach is not only more effective in steering features, but also produces musically more compelling results.
7 FUTURE WORK
As prior work demonstrates, there are several possible avenues for steering transformers. We have established baseline for compositional steering with bias-tuning and shown negative results on compositional steering with prefix tuning, but believe novel loss functions, architectures, and de- coding strategies could improve these methods. In addition, other methods for steering, such as Plug-and-Play side-tuning or CTRL can be applied to this area with potentially promising results.
This study focused on binary features, but most of those binary features were actually derived from cutoffs on continuous features, like amount of dynamic contrast or percentage pitch increase. Future work could include features which are associated with continuous value from 0 to 1.
8 RELATED LITERATURE
This work would be impossible without the wealth of research on transformer models for sequence generation tasks. Starting with Vaswani et al. (2017a), researchers realized that this paradigm en- ables far more coherent and diverse generation than the recurrent neural networks typically used before (Sherstinsky 2018). Another milestone was the development of GPT-3, which showed that with sufficient size/training data such models could potentially perform few shot learning (Brown) cet al. 2020). Several subsequent papers, however, demonstrated the inadequacy of few-shot learn- ing for many tasks (Perez et al. 2021). A few alternatives to online few-shot learning have since emerged.
Prompt tuning was initially explored ad-hoc in the context of finding ways to produce interesting output by GPT-3, and formalized by Lester al. (2021). While originally prompts were de- signed as tokens from the transformer's vocabulary, subsequent studies generalized them to any embeddings prepended the input (Li & Liang 2021). We extend research into prefix tuning by considering aggregation methods among compositional prefixes.
Feature-wise transformations, i.e. elementwise scaling and/or biasing of features in network based on side-information, have been applied in a wide variety of problem settings-see Dumoulin et al. (2018) for an overview. We draw direct inspiration from BitFit's bias-tuning approach (Ben -Zkee et al. 2021) and cast it as a fcature-wise transformation approach. By factoring the additive pertur- bations Figure 3 into their preceding layers, our bias-tuning implementation can be described as a multi-task variant of BitFit where the parameterizations are tied across features. A key difference is that our feature-specific adaptations are designed to be composable, which to our best knowledge has not yet been explored in the context of large language models althoug compositional adaptations using feature-wise multiplicative interactions have been studied in the context of zero-shot image classification (Purushwalkam et al.) 2019). Similarly, side-tuning with general language-man model features, has shown significant enhancements few-shot learning, but typically not performed compositionally (Zhang al. 2019)
summing task- aasc-specffic features
8

---

Under review as a conference paper at ICLR 2022
Contrastive learning is used in representation learning to train network which maps "similar" (positive) inputs to nearby representations and "dissimilar" (negative) inputs far away from the pos-
itive inputs. See LheKhha et al. (2020) for theoretical framework and overview. In generative
modeling, contrastive divergence (Hinton 2002) was proposed to train Restricted Boltzmann Ma-
chines (Smolensky) 1986), image-to-image translation models (Baek al.] [02T] Park et al. [2020]
Liu et al.] 2021). and conditional (Kang & Park] (2020) and unconditional Volicoeur-Martinge
2018) generative adverarial networks (Goodfellow et al. 2014). Our contrastive formulation differs from previous work in two ways. First, rather than selecting positive and negative examples related to the conditioning signal (musical features in our case) and using the contrastive loss to predict which example "agrees" with the signal, we select positive and negative conditioning signals (i.e. different musical features) and use the contrastive loss to predict which conditioning signal explains the prnte-continuttion pair best. Second, we also treat the absence of conditioning signal (i.e. the original generative model) as a negative conditioning signal, meaning that we want the model conditioned on the "correct" musical features to explain the prime- comnec-contitatioon pair better than the unconditional model.
Other work leveraging language models for multiple tasks include CTRL (Keskar al., 2019) and Plug-and-Play models (Dathathri et al. (020). However, CTRL has the disadvantage that it requires knowing the tasks during the training of the large language model, while Plug-and-Play requires multiple passes through the language model, which can be expensive for sufficiently large models.
Background in controllable deep generative models for music Advances in sequence model-
ing (van den Oord al.
2016 Vaswani et al.] [017b] van den Oord et al.] 2017 Choromanski long-form music generation in both the symbolic domain (Oore et al.
ct al.
2020) has enabled
2020
Huang al.
2019
Payne 2019 Liutkus et al., 2021 Hsiao et al., (021) and the audio do-
main van den Oord et al.
2016 Hawthorne et al. 2019 Dieleman et al. Dhariwal et al. 2020).
Similar to language, researchers in music generation have been adapting these language models to- wards controllable generation, such as by conditioning on one part of a musical piece to complete the rest, such as melody harmonization (Simon al. [0008] Liang 2016) Choi al. 20200, or more generally arbitrary partial score completion (Huang et al. 2017 Hadjeres et al. 2017). Represen- tational learning approaches such as autoencoders (AEs) and variational autoencoders (VAEs) have also been used for steering interpolations or transformations along learned latent dimensions, a low- level disentangled attribute-based dimension such as note density (Roberts et al. 2018 Kawai al, 2020), or a high-level learned dimension such as energy level in mood that is then realized through its mapping to low-level features such as note-density and rhythm (Tan & Herremans] 2020), con-
trolling chord progressions and texture independently (Simon et al. [018]] WWng et al. 2020). or rearranging a piece to have increased polyphony or rhythmic density
(Wu & Yang 2021).
Controllable generative models typically does not offer users full control (i.e. only allows users to specify a small number of low-level or high-level controls, or through an example), while relying on its learned stylistic distribution and/or features encoded from the user-specified template piece to fill in the rest of the musical details. In contrast, traditional constraint satisfaction based music generation systems do not have prior knowledge of the desired stylistic distribution, instead rely on users to specify a large number of musical constraints to guide its search (see Pachht & Roy 200T for a survey). When using the former systems, users may still feel a lack of agency, while the latter can impose a laborious process. Our approach explores the space in between, allowing users to compose multiple features along different musical dimensions for short chunks (similar to constraint specification), while leveraging pretrained transformers' expressiveness to aid users in maintaining coherence in virtuosic long-fon oor-form composition.
CONCLUSION
We have shown that music transformers can be directed towards a specific generative "task" using some of the same methods as natural language transformers. In addition, we have studied compo- sitionality in this domain. Compositionality (including relatively high levels of compositionality) is critical in the music domain (and other domains) if the user wants control over the output. We establish that compositionality is a hard problem, and propose adaptations of several solutions from the literature thas-ttuning and prefix-tuning to address these challenges. We find success with bias- tuning, but not with prefix-tuning While our results are promising, there is clearly significant room for improving on the efficacy of steering a transformer compositionally.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Kyungjune Baek, Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Hyunjung Shim. Rethinking the truly unsupervised image-to-image translation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 14154-14163, 2021.
Elad Ben-Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-effcien fine-tuning for transformer-based masked language-models. ArXiv, abs/2106. 10199, 2021.
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. neural probabilistic language model. The journal of machine learning research, 3:1 137-1155, 2003.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ila Sutskever, and Dario Amodei. Language models few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Asso- ciates, Inc., 2020. URL
Kristy Choi, Curtis Hawthorne, Ian Simon, Monica Dinculescu, and Jesse Engel. Encoding musi- cal style with transformer autoencoders. International Conference on Machine Learning, pp. 1999-1908. PMLR, 2020.
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. Rethinking attention with performers. ICLR, 2020.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, J. Yosin- ski, and Rosanne Liu. Plug and play language models: simple approach to controlled text generation. ArXiv, abs/1912.02164. 2020.
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music. arXiv preprint arXiv:2005.000 00341, 2020.
Sander Dieleman, Aaron van den Oord, and Karen Simonyan. The challenge of realistic music generation: modelling raw audio at scale. In Advances in Neural Information Processing Systems.
Vincent Dumoulin, Ethan Perez, Nathan Schucher, Florian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio. Feature-wise transformations. Distill, 2018. doi: 10.239 1//isttilll0 http::/iill..lllo//efeeeeeeeeeee
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neurul information processing systems, 27, 2014.
Gaëtan Hadjeres, François Pachet, and Frank Nielsen. DeepBach: a steerable model for Bach chorales generation. In International Conference on Machine Learning, pp. 1362-1371, 2017.
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel. and Douglas Eck. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Represen- fations, 2019. URL httphtttps:://penreeet
net/ fett:/oormm:dd=r111a
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):177 2002.
Wen-Y Yi Hsiao, Jen- Yu Liu, Yin-Cheng Yeh, and Yi-Hsuan Yang. Compound word transformer: Learning compose full-song music over dynamic directed hypergraphs. AAAI, 2021.
10

---

Under review as a conference paper at ICLR 2022
Cheng-Zht Zhi Anna Huang, Tim Cooijmans, Adam Roberts, Aaron Courville, and Doug Eck. Coun- terpoint by convolution. In Proceedings of the International Conference on Music Information Retrieval, 2017.
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Curtis Hawthorne, An- drew M. Dai, Matthew D. Hoffman, and Douglas Eck. An improved relative self-attention mech- anism for transformer with application music generation. CoRR, abs/ 1tt//809.000 009.04481, 2018. URL
http:// http:///axiv..// org/ abs
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew Dai, Matthew Hoffman, Monica Dinculescu, and Douglas Eck. Music transformer. In International Conference on Learning Representations, 2019.
Natasha Jaques, Shixiang Gu, Richard E. Turner, and Douglas Eck. Tuning recuirent neural net- works with reinforcement learning. CoRR, abs/1611.02790 2016. URL htttp://arxiv.
org/abs/1611. 02796
Alexia Jolicoeur-Martinaau. The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00733 2018.
Minguk Kang and Jaesik Park. Contragan: Contrastive learning for conditional image generation. arXiv preprint arXiv:2006. 12681, 2020.
Lisa Kawai, Philippe Esling, and Tatsuya Harada. Atributes-aware deep music transformation. In Proceedings of the 21st international society for music information retrieval conference, ismir, 2020.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. CTRL: A conditional transformer language model for controllable generation. CoRR, Abs//199.05888, 2019.
wttp:o/g/////...../..// 05858
Phuc H Khac, Graham Healy, and Alan F Smeaton. Contrastive representation learning: A framework and review. IEEE Access, 2020.
Brian Lester, Rami AI-Rfou, and Noah Constant, The power of scale for parameter-offfcient prompt tuning. CoRR, abs/2104.086// 2021. URL /ttpss////// wwtp:///g///aaaaaaa//ooaa 08691
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. CORR, 0/s/2101_00190 2021. URL
00190
Feynman Liang. Bachbot: Automatic composition in the style of bach chorales. Masters thesis, University of Cambridge, 2016.
Rui Liu, Yixiao Ge, Ching Lam Choi, Xiaogang Wang, and Hongsheng Li. Divco: Diverse con- ditional image synthesis via contrastive generative adversarial network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16377-16386, 2021.
Antoine Liutkus, Ondfej Cıfka, Shib-Lun Wu, Umut Simsekli, Yi-Hsuan Yang, and Gael Richard. Relative positional encoding for transformers with linear complexity. In International Conference on Machine Learning. pp. 7067-7079. PMLR, 2021.
Sageev Oore, Ian Simon, Sander Dieleman, Douglas Eck, and Karen Simonyan. This time with feeling: Learning expressive musical performance. Neural Computing and Applications, 32(4): 955-967, 2020.
François Pachet and Pierre Roy. Musical harmonization with constraints: A survey. Constraints, (1):7-19, 2001.
Taesung Park, Alexei A Efros, Richard Zhang, and Jun Yan Zhu. Contrastive learning for unpaired image-to-image translation. In European Conference Computer Vision, pp. 319-345. Springer, 2020.
Christine Payne. MuseNet. hitps: http:://ppeaaa..cooooooo//aaeeee http://ewww...rrrr/////oaaaooaoa 05-04.
2019. Accessed: 2020-

---

Under review as a conference paper at ICLR 2022
Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. CoRR, abss2100.11447,, 2021. URL https:://xxxi......t.apta// http:://xxxvv.og//o.oooo 11tp:////aa...crrr.a///111111111
Senthil Purushwalkam, Maximilian Nickel, Abhinav Gupta, and Marc' Aurelio Ranzato. Task-driven modular networks for zero-shot compositional learning. In Proceedings of the IEEE/CVF Inter- national Conference Computer Vision, pp. 3593-36900 2019.
Adam Roberts, Jesse Engel, Colin Raffel, Curtis Hawthorne, and Douglas Eck. hierarchical latent vector model for learning long-term structure in music. ICML, 2018.
Alex Sherstinsky. Fundamentals recurrent neural network (RNN) and long short-term memory (LSTM) network. CoRR, abs/1808.00 00331,31,, 2018. URL http://arxivv.zz//bbb///
03314
Ian Simon, Dan Morris, and Sumit Basu. MySong: automatic accompaniment generation for vocal melodies. In Proceedings of the SIGCHI conference on human factors in computing systems.
ACM, 2008.
lan Simon, Adam Roberts, Colin Raffel, Jesse Engel. Curtis Hawthoree, and Douglas Eck. Learning latent space of multitrack measures. arXiv preprint arXtv:1806.00199 2018.
Smolensky. Information Processing in Dynamical Systems: Foundations of Harmony Theory, pp. 194-281 281. MIT Press, Cambridge, MA, USA, 1986. ISBN 026268053X.
Hao Hao Tan and Dorien Herremans. Music fadernets: Controllable music generation based on high-level features via low-level feature modelling. ISMIR, 2020.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. WaveNet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 2016.
Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learn- ing. NeurIPS, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. CoRR, 2017a.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017b.
Ziyu Wang, Dingsu Wang, Yixiao Zhang, and Gus Xia. Learning interpretable representation for controllable polyphonic music generation. ISMIR, 2020.
Shih-Lun Wu and Yi-Hsuan Yang. Musemorphose: Full-song fine-grained music style transfer with just one transformer arXiv preprint 1xXix:1055004000 2021.
Jeffrey O. Zhang, Alexander Sax, Amir Roshan Zamir, Leonidas J. Guibas, and Jitendra Malik. Side-tuning: Network adaptation via additive side networks. CoRR, 135003 2019. URI.
http://arxivv http:///axx...rrg//b/////222 wttp////////....1//313333/
A MUSICAL FEATURES
Note that while some features may appear to be opposites (e.g., "loud" vs "soft"), and while it is true that they are mutually exclusive, in fact it is possible for sequence satisfy neither (e.g., it's in middle dynamic level).
Absolute features:
1. "Loud" minimum velocity is greater than 60
2. "Soft" maximum velocity is less than 60
12

---

Under review as a conference paper at ICLR 2022
3. "Has Dynamic Contrast" Extreme Dynamic Contrast" The sequence has two notes whose velocities differ by more than 30
4. "Extreme Dynamic Contrast" The sequence two notes whose velocities differ by more than 70
5. "All consonances" - the sequence has no dissonances (simultaneous notes with an absolute difference modulo 12 of 1, 2, 10, 11, 6)
6. "Long sharp dissonance" the sequence has sharp dissonance (simultaneous notes being played with a difference of or 11) that lasts for a significant amount of time
7. "Only melody" only single note is playing time
8. "Few onsets" when grouping notes according to attack time, there are few groups per second
9. "Many onsets" when grouping notes according to attack time, there are many groups per second
10. "Blocks of two" there groups of two notes being played simultaneously
11. "Larger blocks" there are blocks of three more notes being played simultaneously
12. "Within single key"- all notes fit within a single major scale
Relative features:
1. "Significantly lower average pitch than prime"
2. "Significantly higher average pitch than prime"
3. "Significantly higher number of grouped attacks than prime"
4. "Significantly lower number of grouped attacks than prime"
5. "Significantly more notes per second than prime"
6. "Significantly fewer notes per second than prime"
7. "Could fit in the same key the prime"
8. "Has 2 or pitch classes (pitch mod 12) which the prime doesn't 't have"
B ADDITIONAL FIGURES
C ON A PER-FEATURE LEVEL, THERE IS A BENEFIT TO INCLUDING AN ADDITIONAL STEERED FEATURE IN CONJUNCTION WITH n OTHER STEERED FEATURES AS OPPOSED TO REJECTION SAMPLING + n STEERED FEATURES UP TO 6.
As seen in Figure 8 the benefit of steering doesn't only from increased likelihood of single feature (which in fact would make the joint likelihood total features higher, but would indicate increased compositionality). Rather, for single constant feature, bias-tuning towards that feature increases steerability even in the presence of 5 other steered features.
13

---

Under review as a conference paper at ICLR 2022
Prefix tuning (Contrastive)
Prefix tuning (Non-contrastive)
Unconditional
Feature
all_consonances
Cofactual
0.51 (0.48, 0.54)
0.31 (0.27, 0.35)
0.37 (0.34, 0.40)
Counterfactual
0.37 (0.34, 0.40)
0.23 (0.19, 0.27)
0.28 (0.25, 0.31)
Random
0.30 (0.26, 0.34)
0.27 (0.23, 0.31)
0.17 (0.14, 0.20)
Cofactual
0.78 (0.75, 0.82)
0.82 (0.79, 0.86)
0.52 (0.48, 0.56)
Counterfactual
0.68 (0.64, 0.72)
0.74 (0.71, 0.78)
0.37 (0.33, 0.41)
Random
0.73 (0.69, 0.77)
0.83 (0.80, 0.87)
0.44 (0.39, 0.48)
exrreme_dynamic_oontas
Cofactual
0.36 (0.32, 0.40)
0.45 (0.40, 0.49)
0.08 (0.06, 0.10)
Counterfactual
0.28 (0.24, 0.32)
0.33 (0.29, 0.37)
0.02 (0.01, 0.04)
Random
0.25 (0.21, 0.28)
0.39 (0.35, 0.44)
0.01 (0.00,
few few_onsets
Cofactual
0.58 (0.54, 0.63)
0.49 (0.45, 0.54)
0.06 (0.04, 0.09)
Counterfactual
0.21 (0.18, 0.25)
0.16 (0.12, 0.19)
0.02 (0.00, 0.03)
Random
0.27 (0.23, 0.31)
0,25 (0.21, 0.29)
0.02 (0.01, 0.03)
long sharp dissonance
Cofactual
0.36 (0.32, 0.40)
0.14 (0.11, 0.17)
(0.02, 0.05)
Counterfactual
0.38 (0.34, 0.43)
0.06 (0.04, 0.08)
0.01 (0.00, 0.02)
Random
0.40 (0.35, 0.44)
0.10 (0.07, 0.12)
0.01 (-0.00, 0.01)
loud
Cofactual
0.87 (0.84, 0.89)
0.84 (0.81, 0.88)
0.79 (0.75, 0.82)
Counterfactual
0.73 (0.69, 0.77)
0.55 (0.51, 0.59)
0.35 (0.31, 0.39)
Random
0.74 (0.70, 0.78)
0.57 (0.53, 0.61)
0.44 (0.40, 0.49)
manyvverticalities
Cofactual
0.82 (0.79, 0.86)
0.78 (0.75, 0.82)
0.21 (0.18, 0.25)
Counterfactual
0.50 (0.45, 0.54)
0.44 (0.39, 0.48)
0.13 (0.10, 0.16)
Random
0.70 (0.66, 0.74)
0.61 (0.57, 0.65)
0.20 (0.16, 0.23)
aneentiallyssame..ke
Cofactual
0.64 (0.60, 0.68)
0.55 (0.51, 0.59)
0.65 (0.61, 0.69)
Counterfactual
0.18 (0.15, 0.21)
0.17 (0.13, 0.20)
0.18 (0.15, 0.22)
Random
0.32 (0.28, 0.36)
0.29 (0.25, 0.33)
0.34 (0.30, 0.38)
relative shift
Cofactual
0.52 (0.48, 0.56)
0.47 (0.43, 0.52)
0.34 (0.30, 0.38)
Counterfactual
0.41 (0.37, 0.45)
0.31 (0.27, 0.35)
0.20 (0.16, 0.23)
Random
0,44 (0.39, 0.48)
0,40 (0.36, 0.44)
0.26 (0.23, 0.30)
significantly.higee www.hhhhhhhheeeeeeeeeeeeeeeeeeee center of mass
Cofactual
0.26 (0.22, 0.29)
0.23 (0.19, 0.26)
0.21 (0.18, 0.25)
Counterfactual
0.17 (0.14, 0.20)
0.13 (0.11, 0.16)
(0.08, 0.13)
Random
0.15 (0.12, 0.18)
0.14 (0.11,
0.14 (0.11, 0.17)
significantly higher note density
Cofactual
0.64 (0.60, 0.68)
0.64 (0.60, 0.69)
0.72 (0.68, 0.76)
Counterfactual
0.50 (0.46, 0.55)
0.50 (0.46, 0.55)
0.61 (0.57, 0.65)
Random
0.52 (0.48, 0.56)
0.57 (0.52, 0.61)
0.59 (0.55, 0.64)
seafficcantyyhhggheee wwwthhiicdetssiiiiii
Cofactual
0,58 (0.54, 0.62)
0.59 (0.55, 0.63)
0.58 (0.54,
Counterfactual
0.40 (0.36, 0.44)
0.47 (0.42, 0.51)
0.46 (0.41,
Random
0.42 (0.38, 0.46)
0.46 (0.42, 0.51)
0.49 (0.45, 0.53)
laatecanttyylwwerecem wew.eeeeeerrorreeeeeeee _o mass
Cofactual
0.29 (0.25, 0.33)
0.25 (0.21, 0.29)
0.19 (0.16, 0.22)
Counterfactual
0.22 (0.18, 0.26)
0.15 (0.12, 0.18)
(0.09, 0.14)
Random
0.22 (0.19, 0.26)
0.18 (0.15, 0.21)
0.11 (0.08, 0.13)
signiicantlyLoovernote..de
Cofactual
0.62 (0.57, 0.66)
0.58 (0.54, 0.62)
0..4
Counterfactual
0.29 (0.25, 0.33)
0.29 (0.25, 0.33)
0.02 (0.00, 0.03)
Random
0.36 (0.32, 0.40)
0.37 (0.32, 0.41)
0.06 (0.04, 0.08)
significantly lower rhythmic density
Cofactual
0.58 (0.54, 0.62)
0.51 (0.47, 0.55)
0.23 (0.19,
Counterfactual
0.17 (0.14, 0.20)
0.11 (0.09, 0.14)
0.02 (0.00, 0.03)
Random
0.20 (0.16, 0.23)
0.20 (0.16, 0.23)
0.06 (0.04, 0.08)
soft
Cofactual
0.58 (0.54, 0.62)
0.55 (0.51, 0.59)
0.32 (0.28, 0.36)
Counterfactual
0.26 (0.22, 0.30)
0.18 (0.15, 0.21)
0.04 (0.03, 0.06)
Random
0.28 (0.24, 0.32)
0.21 (0.18, 0.25)
0.08 (0.06, 0.11)
some verticalities
Cofactual
0.92 (0.90, 0.95)
0.91 (0.89, 0.94)
0.74 (0.71, 0.78)
Counterfactual
0.81 (0.78, 0.84)
0.74 (0.70, 0.77)
0.64 (0.60, 0.68)
Random
0.90 (0.87, 0.92)
0.87 (0.84, 0.90)
0.76 (0.72, 0.80)
within biaiin-singl_key
Cofactual
0.62 (0.58, 0.66)
0.58 (0.54, 0.62)
0.63 (0.59, 0.67)
Counterfactual
0.53 (0.48, 0.57)
0,42 (0.37, 0.46)
0.54 (0.50, 0.58)
Random
0.59 (0.55, 0.63)
0.53 (0.49, 0.57)
0.60 (0.55, 0.64)
Table 2: Efficacy comparison across all features.
14

---

Under review as a conference paper at ICLR 2022
0.8
0.7
0.6
Counterfactual
0.5
0.4
0.3
0.2
0.1
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Cofactual
Figure 6: Varying difficulties of features, evaluated with the unconditional model.
Counterfactual
Cofactual
Random
secuve desuntt
preat
Beon-cottaative)
Figure 7: Bar plots.
15

---

Under review as a conference paper at ICLR 2022
Efficacy of single additional steered feature after 5 are added
Bias Tuning Random Prefixes
Bias Tuning Same Prelixes Control Prefix
1.00
0.75
0.50
0.25
0:00
all consonartee
contrasting. dy
extremee dynaa...
few onsets
lmng shop di
many veriicali
loud
potentaaly. SB
relatve DC shift
significntlly fi.
significantl It.
sinifeeanly Ht.
signifccatly Jo
significnnly. Jo..
signfficatly. to.
some verreatl..
soft
within single
Figure 8: Additional gain after 5 features already included occur along almost every feature, suggesting that the tine- transformer really can be used compositionally.
16


Reviews:
[{"review_id": "r3TX7CWmTS", "reviewer": "Reviewer_qjek", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I like the CSP setting proposed in Section 2. Much of the related work on steering focuses on soft, differentiable constraints, for example, generating samples that are likely under a classifier model (Dathathri et al., 2020). This is an interesting alternative perspective.\n\nThe contrastive loss (Section 3.1) and additive model for feature composition (Section 3.2) are clearly described and interesting. However, the paper is missing a discussion of related work on contrastive losses. How does the contrastive loss proposed here compare to other contrastive losses? I am not very familiar with the literature on contrastive losses, and this lack of discussion leaves me unclear about the extent of the novelty/contribution to the contrastive modeling literature.\n\nI am a little confused by Figure 3: it appears from this figure that prefix-tuning and bias-tuning are used together, but later in the paper it seems that these two techniques are analyzed separately?\n\nI found the discussion of the empirical evaluation (Sections 4, 5, and 6) very confusing. Below are some specific questions and concerns.\n\n(1) Section 4.1 discusses fine-tuning the transformer to the MAESTRO dataset. Does this imply that the original transformer is trained on a different dataset? Which one?\n\n(2) Section 5.2 introduces a lot of terminology with in-line definitions that are difficult to follow. I found it difficult to understand what is meant by \"cofactual primes\" and \"counterfactual primes.\" What is the distinction made between the \"maestro validation dataset\" and the \"actual validation dataset\"? Later, terms \"cofactual\"/\"counterfactual\" tuning (Figure 4) and \"cofactual steering\" (Section 6) are introduced without definition and further confuse me.\n\n(3) Also in Section 5.2: does \"efficacy\" mean the probability that a sample satisfies the desired constraints?\n\n(4) In Figure 4, what is the y-axis of the contrastive (right-hand) plot? It seems to carry the same \"MLE Loss\" as the left-hand plot. I'm very unclear what is being shown here (this could be related to my confusion about the meanings of \"cofactual\" and \"counterfactual\" tuning).\n\n(5) How are the error bars (confidence intervals?) in Table 1 and Figure 5 computed? How many samples are used to calculate efficacy?\n\n(6) How should I interpret the results of the listening test in Section 6. The motivation for this work stated in the introduction is that rejection sampling is slow, and we want to fine-tune to get a better proposal distribution that allows us to sample more efficiently. But the listening test finds that fine-tuning produces higher quality samples than rejection sampling. This makes me uncomfortable with the framing in the introduction.\n\nOverall, regarding the experiments. There does appear to be good empirical evidence here to support the advantage of the proposed contrastive loss over MLE. But there does not seem to be an empirical analysis of the additive approximation proposed in Section 3. Nor is there a clear analysis of prefix-tuning versus bias-tuning.\n\nMinor comment: the references section cites many arxiv pre-prints, rather than conference versions of the papers. In many cases these papers have been published for several years, and citing the pre-print versions seems a little sloppy. ", "summary_of_the_paper": "This work considers how to control sampling from transformer-based autoregressive generative models of music. In particular, it considers constraint satisfaction problems (CSP) given a collection of binary constraints. The technical contributions of this paper are (1) a method for fine-tuning (bias-tuning) a model using a contrastive loss for adaptation to a given CSP (Section 3.1) and (2) an additive approximation to bias-tuning for satisfying combinatorial constraints (Section 3.2).", "summary_of_the_review": "I found this paper thought-provoking and creative, but I found the empirical analysis confusing and incomplete. I like the CSP setting outlined in Section 2, and the methods introduced in Section 3 are interesting. However, I found the writing in Sections 4 and 5 confusing to the point that it was difficult for me to understand the empirical results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "UXebL9ecoI", "reviewer": "Reviewer_ZYeZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper tackles an important topic of music generation in the context of Human-AI co-creation. Steering the generation results to the direction intended by the user has been emphasized as a lesson of recent AI song contest events. The authors address the issue focusing on continued generation given a prime. The proposed contrastive learning is novel and the superiority is explained through the experiment. The demo examples support the effectiveness in musicality. However, the presentation and results are not clear to me in term of validating steerability of the model. \n\n- While the demo examples contrasts the superiority of the proposed approach well, I have a bit doubt about the results. The music transformer model is known to tackle the long-term dependency issues in music generation. Using the transformer architecture and relative positional encoding, the model is known to learn wider music contexts and generate consistent note patterns given a prime. However, the majority of unconditional generation examples from Music Transformer diverge in the continuation. On the other hand, the bias-tuning model sounds very consistent between the primes and the continuations. This is somewhat confusing because I was expecting that Music Transform is good at consistent generation whereas bias-tuning may steer the generation to different textures which could not be consistent with the prime, for example, when block chord is chosen as a musical feature for the continuation and the prime does not have block chords. I think the authors should put \"consistent generation\" in the thread of discussion and clarify how steering is different from it. \n\n- The listening test compares unconditional generation to conditional generation with \"randomly chosen three-feature sets\".  This experiment does not explicitly validate the controllability of the model. That is, it does not test steering the generation by the user. I think, if the purpose was rating the preference,  general music listeners could have done it instead of musicians. The user test should be designed such that the musicians choose one of the musical features that they want to control and then rate if the conditional generation is satisfactory or aligned to their expectation. \n\n- \"Non-contrastive\" loss is compared to the proposed contrastive loss in Figure 4. But, it seems that the non-contrastive loss is not clearly defined in the paper. Is it the loss that does not include the negative case (this is my guessing)? \n \n- 12 absolute musical features and 8 relative musical features are defined in Appendix A. I see that they are defined based on music elements such as dynamics, tonality, polyphony, note density and so on. However, they are mostly low-level attributes and are not a complete list of musical elements. Furthermore, more high-level features are possible, for example, syncopation, chromaticism, more jazz chords (e.g, more 7ths chords) but they are not addressed. Authors should clarify the basis of selecting the music features and the scope of possible musical steering with the features.  \n\n- This motivation and necessity of the research was well addressed in the introduction. However, the author missed the following paper which testifies the labor-intensive rejection sampling in the real-world music generation scenario.\n\n** Human-AI Co-creation in Songwriting, Cheng-Zhi Anna Huang, Hendrik Vincent Koops, Ed Newton-Rex, Monica Dinculescu, Carrie Cai, ISMIR 2020.\n\nThere will be more papers that reported similar issues in the context of computational creativity. \n\n< Minor parts>\n- page 2: \"... for instance, 6, the requested... \" --> \"6\" seems to be an errata. \n- page 3: the green symbols (the check marks) in section 3.1 are not visible when it is printed in black and white.\n- page 4: \"prepreding\" --> This looks like a typo. \"prepending\" might be correct\n", "summary_of_the_paper": "This paper presents a model tuning strategy for steering conditional music generation given the pre-trained Music Transformer model. The authors pose the steering as musical feature matching between two sequences from prime and continuation, respectively, and tune the parameters using two methods, prefix tuning and bias-tuning. They show that the latter is more effective for steering conditional music generation. They also present a differentiable proxy of music feature matching considering the positive, negative, and non-adaptive cases. The contrastive loss outperforms the non-contrastive loss and bias-tuning is more effective than prefix tuning in the experiment. ", "summary_of_the_review": "This paper tackled an important issue of music generation. They proved the proposed contrastive loss by compared it to the non-contrastive loss. However, while the aim of the research is providing controllability or steerability of the model by musical features, the listening test seems to validate preference of the continuation. In other words, the listening test does not reflect the control by the musician subjects. The authors should clarify this during the discussion phase. \n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MBmTl5iR3Bo", "reviewer": "Reviewer_LG3E", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "My overall sense of this paper is positive, though I am not deeply familiar with the related work on which it builds.\nThe basic ideas are explained well enough with reasonable motivation and technical justification.\nThe results seem good, although I'm not sure that I fully understand what is being presented.\nIn general, the exposition of the methods is clear, but the evaluations (tables and figures) seem unnecessarily terse and difficult to parse.\nPutting a bit more explanatory text into the captions and surrounding text would significantly improve this paper. Specifically:\n\n- Figure 4 is described in a one-sentence subsection (5.1), which does not at all explain what is being measured in the figure or how to interpret the plots.  The terms \"cofactual\" and \"counterfactual\" are not even introduced until 5.2, but are used in the caption of figure 4 without explanation.  If I understand the plot correctly, it demonstrates that contrastive training eventually assigns high conditional likelihood to \"good\" continuations (blue) and low likelihood to \"bad\" ones (green), and therefore is able to discriminate between them?  \n\n- The term \"efficacy\" is used without an explicit definition, and forms the basis of most of the quantitative evaluations (Table 1, Figure 5).  It was only after looking at the page of example outputs that it is clearly defined as the number of rejected samples prior to generating a positive example.  Please define your terms and metrics!\n\n- Table 1 lists what appears to be mean efficacy scores, as well as some kind of bounds in parentheses?  Are these quantiles?\n\n- Figure 5 seems to reuse the same color-map in multiple ways that are inconsistent between plots, making it needlessly difficult to interpret.  (E.g., \"unconditional\" is blue in the left and green in the middle/right.)\n\n\nFinally, this is a very minor point, but the neon-green checkmark (e.g., in equation 5) is nearly impossible to read against a white background.\nPlease do your readers a favor and stick to high-contrast colors.\n\n", "summary_of_the_paper": "This paper describes a method for adapting the parameters of a symbolic music generation model such that generated continuations of a prefix are more likely to satisfy a set of user-specified constraints.\nThe method builds on prior work by supporting conjunctions (compositions) of constraints, which are treated by embedding constraint specifications as vectors which are provided as input to a bias tuning layer; multiple constraints are handled by averaging their predicted model outputs together.\n(A prefix-tuning method is also investigated, but consistently outperformed by the bias-tuning method.)\nThe model is fine-tuned with a contrastive loss that attempts to maximize the likelihood of continuations that satisfy the supplied constraints while minimizing the likelihood of continuations that violate constraints.\nThe proposed method is evaluated by its \"efficacy\", that is, efficiency of generating valid continuations relative to the unconditional model.\nA human subject listening test demonstrates that the proposed method is more likely to generate \"more musical\" continuations than the unconditional model.\n\n", "summary_of_the_review": "It seems like a nice contribution.  It's confusing in parts, but the issues should be easily fixable.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GEShAzi6OGq", "reviewer": "Reviewer_ezPb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper introduces a novel objective to finetune a model on several features (the contrastive loss of Eq. 5) and compares two existing methods (prefix tuning and bias-tuning) to adapt such a pretrained model so that it generates sequences featuring some user-defined musical features. In particular, these two approaches are compared on two different regimes: a single-feature regime where only one musical feature can be selected, and a compositional setting where multiple musical features can be selected at the same time.\nIf the introduction of the objective of Eq. 6 looks appealing and the two compared methods extracted from recent works, the musical results are not convincing.\nThe proposed task seems interesting from the machine-learning point of view but looks quite artificial from a musical point of view and it's hard to believe that the musical features of Sect. A may be relevant for composers. \n\nThe bibliography seems a bit shallow regarding the relevant literature on generative models for music with (and without) control.\nOverall, the paper is well written even if the clarity could be improved at some places (notations in Eq 2 and 3 are not particularly useful, in Fig. 2, there must be a \\phi_i as an input in order to compute \\ell_x, notations \\ell_true and \\ell_false are more confusing than helpful as they elude the dependence on \\theta_i and \\theta_j, a small table showing the difference during training between the single-feature and compositional regimes could help).\n\nQuestion:\n- Influence of alpha on the efficacy of rejection sampling?", "summary_of_the_paper": "The authors propose to investigate two methods (prefix tuning and bias-tuning) to steer a pretrained music transformer using a set of manually-chosen musical features. The contribution consists in adding custom contrastive loss to the maximum-likelihood objective when finetuning, so that features better capture their intended meaning and offer better control.", "summary_of_the_review": "There are interesting aspects in this paper (especially the comparison between bias-tuning and prefix-tuning in the compositional setting) but the overall contribution is not very substantial (only Eq. 5) and its effects not extensively discussed. The musical examples are not well presented (impossible to know which are the selected features, only one example per continuation for each model, not obvious to distinguish the generations from the prefixes, extracts are too short) and fail to convince.\nThe method could be of interest for a musically-focused conference, but it seems that the technical contribution is not strong enough for ICLR 2022. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a compositional approach to (conditionally) steer pre-trained music transformers to the direction intended by the user.  Overall the scores are mostly negative. The reviewers pointed out some interesting aspects of the paper (e.g., using hard binary constraints as opposed to the soft ones, the contrastive approach). However, one common issue shared by all the reviewers is the clarity of the presentation, which led to many reviewers being confused about various aspects of the paper especially the empirical evaluation. The authors did provide a detailed response to address some of the concerns, but to fully address all the points I anticipate it would require quite substantial change to the paper. A couple reviewers also raised the concerns regarding the limited contribution of the paper. Finally, there appears to be some disagreement between the authors and reviewers regarding how to interpret the listening test results. I hope the authors can take the comments into consideration to further improve this paper for the next submission.
Decision: Reject