C OMPOSING
F
EATURES : C OMPOSITIONAL
M
ODEL A UGMENTATION
FOR
S
TEERABILITY
OF
M
USIC
T RANSFORMERS
Anonymous authors
Paper under double-blind review
A BSTRACT
Music is a combinatorial art.
Given a starting sequence, many continuations are possible, yet often only one is written down.
With generative models, we can explore many.
However, ﬁnding a continuation with speciﬁc combinations of features (such as rising pitches, with block chords played in syncopated rhythm) can take many trials.
To tackle the combinatorial nature of composing features, we propose a compositional approach to steering music transformers, building on lightweight ﬁne-tuning methods such as preﬁx tuning and bias tuning.
We introduce a novel contrastive loss function that enables us to steer compositional models over logical features using supervised learning.
We examine the difﬁculty in steering based on whether features musically follow a prime or not, using existing music as a proxy.
We show that with a relatively small number of extra parameters, our method allows bias tuning to perform successful ﬁne-tuning in both the single-feature and compositional setting.
I NTRODUCTION
Recent research has focused on ways of adapting unconditional language models to perform well on conditional tasks they were not initially trained on.
Methods such as ﬁne-tuning, side-tuning, bias-tuning, and prompt or preﬁx tuning have emerged as lead candidates for such tasks such as steering the sentiment of or words mentioned in a sentence (Ben-Zaken et al., 2021; Li & Liang, 2021;
Lester et al., 2021; Dathathri et al., 2020).
However, such tasks are normally presented in a non- or minimally-compositional approach (possibly controlling for two orthogonal variables, such as sentiment and topic (Keskar et al., 2019), but rarely more than that).
In contrast, in the domain of music generation, sequence-continuation is inherently a highly compositional problem: the user likely has many aspects of the output they would like to control, such as speed, dynamics, harmony, or texture, each of which can be decomposed into multiple sub-features.
Furthermore, the relationship between these features and the output is more diffuse than in the NLP settings: while in examples such as Keskar et al.
(2019), individual words indicate the different conditions being satisﬁed, in music the entire sequence of tokens are used in evaluating a single feature (for instance, average pitch over a span of time depends on all tokens representing that span).
This makes compositionality even more challenging than in the text generation setting.
On the other hand, music is a sequential domain where there are clear logical features which can be examined, such as average dynamics or number of notes per second.
This further motivates using music as a test-bed for highly compositional tasks involving simultaneously steering an autoregressive model towards several particular desired attributes according to different classiﬁers.
As a more immediate motivation, consider the following scenario: a composer wants to sample from the pre-trained Music Transformer (Huang et al., 2018) to complete a musical phrase.
In addition to the overall musical quality of the continuation, they want it to stay in key and switch to using block chords (i.e. a few notes played together at once).
Repeatedly sampling continuations from the model and cherry-picking a good sample ( rejection sampling ) would be very labor-intensive, but assuming they can compute binary features for “stays in key” and “uses block chords”, the composer could sample a large number of continuations and cherry-pick from the smaller set of continuations which exhibit all features (thereby delegating part of the accept/reject step).
While signiﬁcantly less labor-intensive, this solution could potentially require sampling enormous amounts of continuations
[Figure]
if, for instance, 6 the requested features signiﬁcantly differ from those of the priming sequence (such that it and the desired continuation form an unlikely sequence according to the generative model).
In other words, the pre-trained model could be a poor proposal distribution for some applications.
In this work, we are interested in developing better proposal distributions through an adaptation approach which can steer the generative model towards continuations which i) are signiﬁcantly more likely to exhibit the requested feature, and ii) exhibit a satisfactory musical quality.
The approach should be able to accommodate a large number of features without adding signiﬁcant memory or computation overhead.
We achieve this by making features composable, making it possible to steer features independently and also multiple features at once.
Figure 1 shows that when using a pretrained transformer model augmented with a relatively small number of additional parameters, we are able to steer towards arbitrary logical music features and achieve realistic music generation simply by sampling directly from the model 1 .
In contrast, the same unconditional transformer model fails to produce any examples satisfying those features even when generating 100 samples.
With our method, a composer can control both the short-chunk features, and the long-form structure, by chaining together chunks steered in different directions (using different combinations of features), while maintaining long-term coherence (by leveraging the transformers’ full self-attention receptive ﬁeld).
See the demo in the footnote for longer steered examples, and compositions semi-automatically generated using our feature tuning approach in a musician-directed way.
P ROBLEM FORMULATION
Music Transformer is an autoregressive language model which decomposes the joint probability of a sequence of tokens x 1 , . . .
, x N (where x n ∈ K , and K is a set of categorical tokens) into
[Equation]
It leverages a common modeling approach which represents the conditional probabilities p ( x n | x 1 , . . .
, x n − 1 ) using a neural network (Bengio et al., 2003).
As its name implies, Music Transformer uses a Transformer network architecture (Vaswani et al., 2017a).
Each token x n is ﬁrst mapped to a real-valued embedding e n (for instance using a lookup table), then the network maps each sequence e 1 , . .
, e n − 1 to a probability distribution for the value of x n over the elements of K .
Sequence continuation in an autoregressive language model works by repeatedly sampling from its distribution over the next token given the previous tokens.
Starting from some priming se-
quence x p = ( x 0 , . . .
, x M ) , we ﬁrst sample y M +1 ∼ p ( · | x 0 , . . .
, x M ) , then y M +2 ∼ p ( · | x 0 , . . .
, x M , y M +1 ) , and so on, until the end of the continued sequence x c = ( y M +1 , . . .
, y N ) .
Many downstream tasks can be cast as sequence continuation problems, including the steerable music generation problem investigated in this work.
We are given a set of features Φ = { φ j } Jj =1 , φ j ∈ K N → { 0 , 1 } .
Each φ j takes the value 1 if a prime-continuation pair exhibits that feature (which we note as ( x p , x c ) | = φ j ), and 0 otherwise.
Note that the features must take both prime and continuation sequences as input, since some continuation features may be relative to the priming sequence (e.g. “signiﬁcantly higher pitch”).
Our true objective with respect to feature φ j is to steer the model towards a distribution which maximizes
[Equation]
while maintaining musicality.
This objective is non-differentiable because ( x p , x c ) | =
φ j is a nondifferentiable satisﬁability criterion.
In addition to the single-feature problem, we also consider the problem of composed features ˆΦ , i.e.
[Equation]
to account for scenarios where a user is interested in steering the model towards multiple features (such as in the “stays in key” and “exhibits extreme dynamical contrast” scenario dicussed in the introduction).
P ROPOSED APPROACH
We start by describing our proposed approach in the single-feature case and later on explain how we adapt it to the compositional case.
3. L IKELIHOOD - BASED TRAINING
While approaches using reinforcement learning—such as KL-regularized deep Q-learning (Jaques et al., 2016)—could be used to overcome the non-differentiability problem, in this work we consider a proxy loss in the form of the negative log-likelihood
[Equation]
which we use in two ways:
Positively : given a prime–continuation pair ( x p , x c ) | = φ j , we ﬁnd an adaptation θ j of the model’s parameters θ that minimizes l (cid:88) (we use the symbol (cid:88) to denote the fact that l is computed using the correct parameters θ j ).
By using prime–continuation examples that sound musical, we ensure that the steered model stays musically grounded.
Negatively : we can also take advantage of other features φ i for which ( x p , x c ) (cid:54)| =
φ i , by maximizing l × (we use the symbol × to denote the fact that l is computed using the incorrect parameters θ i ).
The positive case corresponds to maximum-likelihood training.
Additionally, we can exploit the intuition that the adapted parameters θ j should “explain” the prime–continuation pair ( x p , x c ) | =
φ j better than θ i (for some feature φ i for which ( x p , x c ) (cid:54)| = φ i ) or θ (the non-adapted model parameters, with a corresponding loss l ∅ ).
In other words, we can maximize the probability of choosing θ j over θ i and θ by minimizing a contrastive loss of the form
[Figure]
[Equation]
We propose a loss that interpolates between Equations 4 and 5 using an α coefﬁcient (which is treated as a hyperparameter):
[Equation]
Intuitively, the maximum-likelihood setting should sufﬁce to achieve our adaptation goals, but in practice we ﬁnd that the approach beneﬁts from the inclusion of negative cases through a contrastive loss term.
We tried different α values and found α = 0 .
8 to work well in practice.
See Figure 2 for an illustration of the training setup.
3. F EATURE - CONDITIONAL ADAPTATION
Fine-tuning all model parameters can be prohibitive if the number J of features is large (let alone combinatorially large in the compositional case); however, recent work provides effective and lightweight alternatives:
Preﬁx tuning (Li & Liang, 2021) works by preprending learnable task embeddings
e − K , . . .
, e − 1 to the priming sequence embeddings e 1 , . . .
, e M .
The loss gradient is then backpropagated through the language model and into the task embeddings.
Bias-tuning (Ben-Zaken et al., 2021) works by adapting a small subset θ b ⊂ θ of the transformer’s parameters, namely the biases of its afﬁne transformations.
Since these biases amount to a small fraction of the model’s parameters, in the case where the number of tasks is relatively small, tuning separate θ b for each task becomes feasible.
We present an extension to bias-tuning where the number of tasks is exponential in the number of total classiﬁcation functions, using an approach which nevertheless only requires a number of tuned parameters linear in the number of total classiﬁcation functions.
In practice, while preﬁx tuning showed promise in the single-feature setting, we were unable to make it work in the compositional setting.
We therefore focus our investigation on bias-tuning and
present
preﬁx tuning results in the Appendix.
In the compositional setting, a naive approach requires learning 2 | Φ | − 1 model adaptations.
Instead, we propose to express the adaptation for a composed feature ˆΦ as the combination of the θ j of its underlying features φ j ∈ ˆ Φ .
More speciﬁcally, for bias-tuning we average the adapted biases as
E m b .
MLP MLP
MLP MLP
Bias vector
E m b .
J
Concat/Average
Prefixtuning
Average
MLP MLP
Bias vector
Average
Bias tuning + + +
Output Input
E m b e dd i n g l a y e r
I nn e r l a y e r I nn e r l a y e r
[Equation]
Note that we do not simply use the above heuristic to compose feature adaptations post-hoc ; we train the model in a compositional setting (by sampling prime–continuation pairs ( x p , x c ) | = ˆΦ for various composed features) so that the single-feature adaptations can learn to work well in conjunction with each other.
See Figure 3 for an illustration of the methods.
M ETHODOLOGY 4. M USICAL F EATURES
We consider 18 features representing properties that a composer would typically like to control; see the Appendix for a complete list.
Features include both absolute and relative features.
Absolute features apply only to the continuation, while relative features, such as “the continuation is signiﬁcantly higher in pitch than the prime” or “the continuation is signiﬁcantly more rhythmically dense than the prime” describe relations between the prime and continuation.
Note that this type of model should work with any feature function which takes in sequences and returns a boolean; we chose examples that have clear musical meanings and are easy to implement.
In addition to exploring binary musical features, we also explore ﬁne-tuning to a provided dataset; in our experiments we ﬁne-tune to the Maestro dataset (Hawthorne et al., 2019), and refer to this feature as φ (cid:122) .
4. T RAINING DATA
The prime–continuation pairs used to train the feature-speciﬁc adaptations are drawn from the Maestro dataset (Hawthorne et al., 2019), an open-source collection of virtuoso piano performances.
Each prime is 100 tokens long (approximately 2-10 seconds), and each continuation is alsotokens long.
Note that since all prime–continuation pairs are drawn from the Maestro dataset, we implicitly assume that ( x p , x c ) | =
φ (cid:122) always holds.
In the single-preﬁx setting, when a prime–continuation pair ( x p , x c ) is sampled for feature φ j , we also select another feature φ i at random such that ( x p , x c ) (cid:54)| = φ i and use its corresponding θ i in
Equation 6.
In the compositional case, we ﬁrst extend Φ so that each feature has a corresponding “negated” feature (e.g. “has both loud and soft pitches” vs “no contrast in dynamics”), bringing the total
[Figure]
Contrastive number of features to 2 · | Φ | = 36 .
By deﬁnition, each prime–continuation pair exhibits exactlyfeatures.
When sampling a pair ( x p , x c ) , we compute l (cid:88) with respect to a random subset of those 18 features (with size drawn at random from U [1 , 12] ).
We compute l × by negating some of the sampled features.
4. I MPLEMENTATION
In the single-feature case, we use a standard preﬁx tuning model as per Li & Liang (2021), associating every feature φ j with a preﬁx v j .
As in their paper, we used the ﬁnding that a relatively low-dimensional embedding space of feature preﬁxes (200 dimensions) projected to a higherdimensional embedding (2048 dimensions) using a shared MLP across feature preﬁxes worked well.
The pre-trained transformer model had no parameters modiﬁed, but was changed to accept these 2048-dimensional vectors as 4 512-dimensional vectors prepended before the 512-dimensional vectors representing the input tokens (as the transformer’s latent embedding was 512 dimensions).
The resulting sequence of vectors were masked using causal attention in order to predict x c .
The loss used in the compositional setting is almost identical to Equation 6, the main difference being in the contrastive loss implementation.
The features used to compute l × are obtained by negating some of the features used to compute l (cid:88) , and the fraction β of negated features is used to modulate the interpolation coefﬁcient as α (cid:48)
=
α − min( α, β ) .
The intuition is that the differences in steering should be more noticeable when comparing feature sets with less overlap.
R ESULTS — SINGLE - FEATURE REGIME 5. T RAINING STATISTICS
As seen in Figure 4, in the contrastive case the model not only learns to assign progressively more weight to ( x p , x c | v j ) (the “positive” examples), but also learns to assign progressively less weight to ( x p , x c | v i )
(the “negative” examples).
5. T HE LARGE VARIANCE IN DIFFICULTY OF PRIME - FEATURE PAIRS
As shown in Figure 6, there is a large difference in the efﬁcacy of rejection sampling with respect to primes from the Maestro validation dataset which are followed in the Maestro validation dataset by a satisfying continuation (here termed “cofactual primes”), and rejection sampling with respect to primes from the maestro validation dataset which in fact are not followed by a satisfying continuation (“counterfactual primes”).
In fact, for some continuations the “counterfactual” probability of
[Table]
efﬁcacy is almost 0%.
Preﬁx tuning not only increases the efﬁcacy of steering overall, but also tends to decrease the relative difﬁculty of “counter-factual” steering.
5. C ONTRASTIVE LOSS GENERALLY IMPROVES EFFICACY OVER NON - CONTRASTIVE LOSS
While efﬁcacy differs for different primes and features
[Table]
contrastive learning generally signiﬁcantly outperforms non-contrastive learning, which almost universally outperforms the unconditional model.
As seen in Table
[Table]
it is usually (but not always) the case that the efﬁcacy of the model trained with contrastive loss is signiﬁcantly higher than that trained with non-contrastive loss
[Table]
and very rarely do both contrastive and non-contrastive models fail (i
[Table]
e., fail to perform signiﬁcantly better
[Table]
and actually seem to do worse, than the unconditional model).
[Figure]
[Table]
[Figure]
[Table]
[Figure]
R ESULTS IN THE C OMPOSITIONAL SETTING
Steering unconditionally becomes intractable as the number of steered features increases.
Under a naive assumption of independence among feature presence, one would assume that the rejection sampling efﬁcacy in steering a total of n features would roughly equal p n , where p is the mean probability of a feature being satisﬁed.
While this assumption is naive, we do ﬁnd that in the unconditional setting, the level of efﬁcacy of unconditional sampling drops substantially as the number of features to be steered increases.
On average, only 0.2% of the time were 6 cofactually steered features all achieved correctly using rejection sampling.
Trained models degrade at a much slower rate as the number of steered features increases.
While the bias-tuning model had average efﬁcacy of 59% in the single preﬁx setting with randomly sampled preﬁxes, it dropped by an order of magnitude (to 7.9%) in the randomly-sampled 6-preﬁx setting (still signiﬁcantly outperforming unconditional rejection sampling).
Bias-tuning is the most effective way to steer a model as the number of steered features increases.
This was true both for “cofactual” steered features and random steered features (although random steered features understandably was a harder task overall).
Notably, even though preﬁx tuning had performed well when trained on single preﬁxes, it failed to generalize to the compositionallytrained case even in the cases that a single preﬁx was being steered.
Listening test
The above experiments evaluate the effectiveness of methods in satisfying the requested features.
To evaluate the musical quality of these ‘successful’ generations, we conducted a listening study with musicians to compare the steering of three features between our overall best augmentation approach (i.e. bias tuning) and the baseline of rejection sampling on the unconditional model.
Questions were posed as pairwise comparisons, and listeners were asked to rate which one they thought was more musical.
To prepare the samples, we randomly picked 120 primes from the test set.
For steering each prime, we randomly chose three-feature sets that were present in the dataset as the conditioning features.
We asked eight musicians to each rate ﬁfteen pairs.
The results show that our bias-tuning approach was much preferred over the unconditional, and the results were statistically signiﬁcant ( p < 0 . 0003 ).
Bias tuning won 63 of the pairwise comparisons, tied forpairs, and lost for 31 pairs.
This shows that our bias-tuning approach is not only more effective in steering features, but also produces musically more compelling results.
F UTURE W ORK
As prior work demonstrates, there are several possible avenues for steering transformers.
We have established a baseline for compositional steering with bias-tuning and shown negative results on compositional steering with preﬁx tuning, but believe novel loss functions, architectures, and decoding strategies could improve on these methods.
In addition, other methods for steering, such as
Plug-and-Play, side-tuning or CTRL can be applied to this area with potentially promising results.
This study focused on binary features, but most of those binary features were actually derived from cutoffs on continuous features, like amount of dynamic contrast or percentage pitch increase.
Future work could include features which are associated with a continuous value from 0 to 1.
R ELATED L ITERATURE
This work would be impossible without the wealth of research on transformer models for sequence generation tasks.
Starting with Vaswani et al.
(2017a), researchers realized that this paradigm enables far more coherent and diverse generation than the recurrent neural networks typically used before (Sherstinsky, 2018).
Another milestone was the development of GPT-3, which showed that with sufﬁcient size/training data such models could potentially perform few shot learning (Brown et al., 2020).
Several subsequent papers, however, demonstrated the inadequacy of few-shot learning for many tasks (Perez et al., 2021).
A few alternatives to online few-shot learning have since emerged.
Prompt tuning was initially explored ad-hoc in the context of ﬁnding ways to produce interesting output by GPT-3, and was formalized by Lester et al.
While originally prompts were designed as tokens from the transformer’s vocabulary, subsequent studies generalized them to any embeddings prepended to the input (Li & Liang, 2021).
We extend research into preﬁx tuning by considering aggregation methods among compositional preﬁxes.
Feature-wise transformations, i.e. elementwise scaling and/or biasing of features in a network based on side-information, have been applied in a wide variety of problem settings—see Dumoulin et al.
(2018) for an overview.
We draw direct inspiration from BitFit’s bias-tuning approach (Ben-Zaken et al., 2021) and cast it as a feature-wise transformation approach.
By factoring the additive perturbations in Figure 3 into their preceding layers, our bias-tuning implementation can be described as a multi-task variant of BitFit where the parameterizations are tied across features.
A key difference is that our feature-speciﬁc adaptations are designed to be composable, which to our best knowledge has not yet been explored in the context of large language models—although compositional adaptations using feature-wise multiplicative interactions have been studied in the context of zero-shot image classiﬁcation (Purushwalkam et al., 2019).
Similarly, side-tuning, or summing task-speciﬁc features with general language-model features, has shown signiﬁcant enhancements in few-shot learning, but is typically not performed compositionally (Zhang et al., 2019)
Contrastive learning is used in representation learning to train a network which maps “similar” (positive) inputs to nearby representations and “dissimilar” (negative) inputs far away from the positive inputs.
See Le-Khac et al.
(2020) for a theoretical framework and overview.
In generative modeling, contrastive divergence (Hinton, 2002) was proposed to train Restricted Boltzmann Machines (Smolensky, 1986), image-to-image translation models (Baek et al., 2021; Park et al., 2020;
Liu et al., 2021), and conditional (Kang & Park, 2020) and unconditional (Jolicoeur-Martineau, 2018) generative adverarial networks (Goodfellow et al., 2014).
Our contrastive formulation differs from previous work in two ways.
First, rather than selecting positive and negative examples related to the conditioning signal (musical features in our case) and using the contrastive loss to predict which example “agrees” with the signal, we select positive and negative conditioning signals (i.e. different musical features) and use the contrastive loss to predict which conditioning signal explains the prime–continuation pair best.
Second, we also treat the absence of a conditioning signal (i.e. the original generative model) as a negative conditioning signal, meaning that we want the model conditioned on the “correct” musical features to explain the prime–continuation pair better than the unconditional model.
Other work leveraging language models for multiple tasks include CTRL (Keskar et al., 2019) and
Plug-and-Play models (Dathathri et al., 2020).
However, CTRL has the disadvantage that it requires knowing the tasks during the training of the large language model, while Plug-and-Play requires multiple passes through the language model, which can be expensive for sufﬁciently large models.
Background in controllable deep generative models for music Advances in sequence modeling (van den Oord et al., 2016; Vaswani et al., 2017b; van den Oord et al., 2017; Choromanski et al., 2020) has enabled long-form music generation in both the symbolic domain (Oore et al., 2020;
Huang et al., 2019; Payne, 2019; Liutkus et al., 2021; Hsiao et al., 2021) and the audio domain (van den Oord et al., 2016; Hawthorne et al., 2019; Dieleman et al.
; Dhariwal et al., 2020).
Similar to language, researchers in music generation have been adapting these language models towards controllable generation, such as by conditioning on one part of a musical piece to complete the rest, such as melody harmonization (Simon et al., 2008; Liang, 2016; Choi et al., 2020), or more generally arbitrary partial score completion (Huang et al., 2017; Hadjeres et al., 2017).
Representational learning approaches such as autoencoders (AEs) and variational autoencoders (VAEs) have also been used for steering interpolations or transformations along learned latent dimensions, a lowlevel disentangled attribute-based dimension such as note density (Roberts et al., 2018; Kawai et al., 2020), or a high-level learned dimension such as energy level in mood that is then realized through its mapping to low-level features such as note-density and rhythm (Tan & Herremans, 2020), controlling chord progressions and texture independently (Simon et al., 2018; Wang et al., 2020), or rearranging a piece to have increased polyphony or rhythmic density (Wu & Yang, 2021).
Controllable generative models typically does not offer users full control (i.e. only allows users to specify a small number of low-level or high-level controls, or through an example), while relying on its learned stylistic distribution and/or features encoded from the user-speciﬁed template piece to ﬁll in the rest of the musical details.
In contrast, traditional constraint satisfaction based music generation systems do not have prior knowledge of the desired stylistic distribution, instead rely on users to specify a large number of musical constraints to guide its search (see Pachet & Roy (2001) for a survey).
When using the former systems, users may still feel a lack of agency, while the latter can impose a laborious process.
Our approach explores the space in between, allowing users to compose multiple features along different musical dimensions for short chunks (similar to constraint speciﬁcation), while leveraging pretrained transformers‘ expressiveness to aid users in maintaining coherence in long-form composition.
C ONCLUSION
We have shown that music transformers can be directed towards a speciﬁc generative “task” using some of the same methods as natural language transformers.
In addition, we have studied compositionality in this domain.
Compositionality (including relatively high levels of compositionality) is critical in the music domain (and other domains) if the user wants control over the output.
We establish that compositionality is a hard problem, and propose adaptations of several solutions from the literature (bias-tuning and preﬁx-tuning) to address these challenges.
We ﬁnd success with biastuning, but not with preﬁx-tuning.
While our results are promising, there is clearly signiﬁcant room for improving on the efﬁcacy of steering a transformer compositionally.
A M USICAL F EATURES
Note that while some features may appear to be opposites (e.g., “loud” vs “soft”) and while it is true that they are mutually exclusive in fact it is possible for a sequence to satisfy neither (e g., if it’s in a middle dynamic level).
Absolute features:
“Loud” - minimum velocity is greater than2.
“Soft” - maximum velocity is less than
“Has Dynamic Contrast” - Extreme Dynamic Contrast” - The sequence has two notes whose velocities differ by more than4.
“Extreme Dynamic Contrast” - The sequence has two notes whose velocities differ by more than5.
“All consonances” - the sequence has no dissonances (simultaneous notes with an absolute difference modulo 12 of 1, 2, 10, 11, or 6) 6.
“Long sharp dissonance” - the sequence has a sharp dissonance (simultaneous notes being played with a difference of 1 or 11) that lasts for a signiﬁcant amount of time 7.
“Only melody” - only a single note is playing at a time 8.
“Few onsets” - when grouping notes according to attack time, there are few groups per second 9.
“Many onsets” - when grouping notes according to attack time, there are many groups per second 10.
“Blocks of two” - there are groups of two notes being played simultaneously 11.
“Larger blocks” - there are blocks of three or more notes being played simultaneously 12.
“Within single key” - all notes ﬁt within a single major scale
Relative features:
“Signiﬁcantly lower average pitch than prime” 2.
“Signiﬁcantly higher average pitch than prime” 3.
“Signiﬁcantly higher number of grouped attacks than prime” 4.
“Signiﬁcantly lower number of grouped attacks than prime” 5.
“Signiﬁcantly more notes per second than prime” 6.
“Signiﬁcantly fewer notes per second than prime” 7.
“Could ﬁt in the same key as the prime” 8.
“Has 2 or more pitch classes (pitch mod 12) which the prime doesn’t have”
B A DDITIONAL FIGURES C O N A PER - FEATURE LEVEL , THERE IS A BENEFIT TO INCLUDING AN ADDITIONAL STEERED FEATURE IN CONJUNCTION WITH n OTHER STEERED FEATURES AS OPPOSED TO REJECTION SAMPLING + nSTEERED FEATURES UP TO n = 6 .
As seen in Figure 8, the beneﬁt of steering doesn’t only come from an increased likelihood of a single feature (which in fact would make the joint likelihood of 6 total features higher, but would not indicate increased compositionality).
Rather, for a single constant feature, bias-tuning towards that feature increases steerability even in the presence of 5 other steered features.
[Table]
signiﬁcantly higher rhythmic density
[Table]
Counterfactual 0.40 (0.36, 0.44) 0.47 (0.42, 0.51) 0.46 (0.41, 0.50)
Random 0.42 (0.38, 0.46) 0.46 (0.42, 0.51) 0.49 (0.45, 0.53) signiﬁcantly lower center of mass
Cofactual 0.29 (0.25, 0.33) 0.25 (0.21, 0.29) 0.19 (0.16, 0.22)
Counterfactual 0.22 (0.18, 0.26) 0.15 (0.12, 0.18) 0.12 (0.09, 0.14)
Random 0.22 (0.19, 0.26) 0.18 (0.15, 0.21) 0.11 (0.08, 0.13) signiﬁcantly lower note density
Cofactual 0.62 (0.57, 0.66) 0.58 (0.54, 0.62) 0.14 (0.11, 0.17)
Counterfactual 0.29 (0.25, 0.33) 0.29 (0.25, 0.33) 0.02 (0.00, 0.03)
Random 0.36 (0.32, 0.40) 0.37 (0.32, 0.41) 0.06 (0.04, 0.08) signiﬁcantly lower rhythmic density
Cofactual 0.58 (0.54, 0.62) 0.51 (0.47, 0.55) 0.23 (0.19, 0.27)
Counterfactual 0.17 (0.14, 0.20) 0.11 (0.09, 0.14) 0.02 (0.00, 0.03)
Random 0.20 (0.16, 0.
[Table]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure]
Cofactual 0.
C o un t e r f a c t u a l a ll _ c
o n s o n a n c e s c o n t r a s t i n g _ d y n a m i
c s e x t r e m e _
d y n a m i
c _
c o n t r a s t f e w _ o n s e t s l o n g _ s h a r p _ d
i s s o n a n c e l o u d m a n y _ v e r t i c a li t i e
s p
o t
e n t
i a ll y _ s a m e _ k e y r e l a t i v e _ p c
s h
i f t s i g n i f i c a n t l y _
h
i g h e r _
c
e n t e r _
o f
_ m a s s s
i g n
i f i c a n t l y _
h
i g h e r _ n o t e _ d e n s i t
y s i g n
i f i c a n t l y _
h
i g h e r _
r h y t
h m i c _ d e n s i t
y s i g n
i f i c a n t l y _ l o w e r _ c e n t e r _ o f _ m a s s s i
g n
i f i c a n t l y _ l o w e r _ n o t e _ d e n s i t
y s i g n
i f i c a n t l y _ l o w e r _ r h y t
h m i c _ d e n s i t y s
[Table]
P e r c e n t s a t i s f
y i n g f
[Table]
a t u r e
[Table]
s i n g l e _ k e y 0.
P e r c e n t s a t i s f
y i n
g f e a t u r e
[Table]
[Table]
Prefix tuning (Non-contrastive) a ll _ c o n s o n a n c e s c o n t r a s t i n g _ d y n a m i c s e x t
r
e m e _
d y n a m i
c _
c o n t r a s t f e w _ o n s e t s l o n g _ s h a r p _ d
i s s o n a n c e l o u d m a n y _ v e r t i c a li t i e
s p
o t
e n t
i a ll y _ s a m e _ k e y r e l a t i v e _ p c
s h
i f t s i g n i f i c a n t l y _
h
i g h e r _
c
e n t e r _
o f
_ m a s s s
i g n
i f i c a n t l y _
h
i g h e r _ n o t e _ d e n s i t
y s i g n
i f i c a n t l y _
h
i g h e r _
r h y t
h m i c _ d e n s i t
y s i g n
i f i c a n t l y _ l o w e r _ c e n t e r _ o f _ m a s s s i
g n
i f i c a n t l y _ l o w e r _ n o t e _ d e n s i t
y s i g n
i f i c a n t l y _ l o w e r _ r h y t
h m i c _ d e n s i
t y s o f t s o m e _ v e r t
i c a li t i e s w i
t h i n _ s i n g l e _ k e y 0.
P e r c e n t s a t i s f
y i n
g f e a t u r e
Random