SONG2QUARTET: A SYSTEM FOR GENERATING STRING QUARTET
COVER SONGS FROM POLYPHONIC AUDIO OF POPULAR MUSIC
Graham Percival, Satoru Fukayama, Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST), Japangraham@percival-music.ca, s.fukayama@aist.go.jp, m.goto@aist.go.jp

ABSTRACT

Target audio file
Pop song

We present Song2Quartet, a system for generating string
quartet versions of popular songs by combining probabilistic models estimated from a corpus of symbolic classicalmusic with the target audio file of any song. Song2Quartet
allows users to add novelty to listening experience of theirfavorite songs and gain familiarity with string quartets. Previous work in automatic arrangement of music only used
symbolic scores to achieve a particular musical style; ourchallenge is to also consider audio features of the target
popular song. In addition to typical audio music content
analysis such as beat and chord estimation, we also use timefrequency spectral analysis in order to better reflect partial
phrases of the song in its cover version. Song2Quartet produces a probabilistic network of possible musical notes at
every sixteenth note for each accompanying instrument of
the quartet by combining beats, chords, and spectrogramfrom the target song with Markov chains estimated fromour corpora of quartet music. As a result, the musical scoreof the cover version can be generated by finding the optimal
paths through these networks. We show that the generated
results follow the conventions of classical string quartet music while retaining some partial phrases and chord voicingsfrom the target audio.
1. INTRODUCTION
Cover songs are arrangements of an original song with certain variations which add novelty. Changing the instruments
used is one such variation, but a complete switch of instrumentation may result in very unusual parts. For example,
completely replacing a chord-heavy guitar part with a violin
may result in unplayable (or very difficult) chords. Arranging music for different instruments requires consideration
about the music those instruments normally perform.
Previous approaches in automated arrangement are mostly
performed in the symbolic domain of music. Melody harmonization and re-harmonization of chord sequences takesymbols of chords or pitches as inputs [1, 7, 10, 16]. Guitar arrangements of piano music can be generated from ac Graham Percival, Satoru Fukayama, Masataka Goto.
Licensed under a Creative Commons Attribution 4.0 International License(CC BY 4.0). Attribution: Graham Percival, Satoru Fukayama, Masataka Goto. “ Song2Quartet: A System for Generating String Quartet CoverSongs from Polyphonic Audio of Popular Music ”, 16th International
Society for Music Information Retrieval Conference, 2015.

114

Corpus of symbolic music
Mozart string quartets

Audio analysis
Spectral magnitude,
onsets, chords, etc.

Score analysis
Probabilistic modelsof pitch and rhythm

Probabilistic
generation
Paths throughnetworks

Score
Mozart quartet-stylecover of pop song

Figure 1: Generating a cover song with a specific style.
Sample results are available at:https://staff.aist.go.jp/m.goto/Song2Quartet/

MusicXML score [14]. Statistical modelling of a corpus
has also been used to generate electronic dance music [6].
Furthermore, automatically generating music in a specificinstrumental style is not well explored. In a great dealof work on computer-assisted composition [8], some automatic composition systems attempted to generate resultswith a particular composer’s musical style [4] or the user’s
musical style [15]. However, those systems cannot be used
to generate cover songs in a particular instrument style bypreserving the recognizable parts of the original songs.
We present Song2Quartet to address this issue. An
overview of our system is shown in Figure 1. Two novel
aspects of this work, the audio analysis for generating coversongs and generating music in a specific instrumental style,are addressed in the audio analysis and score analysis modules, respectively.
To ensure that the generated cover songs include featuresthat are also recognizable in the original audio, the audio
analysis module estimates notable rhythms, chord voicings,
and contrary motions between melody and bass by extracting the audio spectrum. In parallel, to generate music to
be playable and recognizably following the classical string
quartet style, the score analysis module captures characteristics of the string quartet from the corpus of symbolic
music such as the typical note onsets in a measure and thepitch transitions of each instrument in the quartet.
These two aspects are balanced by means of a probabilistic formulation, where the corpus style and audio analysis
are combined by weighted multiplication. The audio analysis provides probabilities for observing note events at every
16th note, and the score analysis mainly provides the transition probabilities of notes. We formalize our generationof cover songs as finding the sequence of notes which maximizes the probabilities obtained from the modules using
dynamic programming, with techniques to compress thesearch space to make our problem tractable.

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015

Onset
estimation

CB

AM

AO

time

M
MIDI

MIDI

MIDI

MIDI
time

C

time

16th -note
rhythm grid1st -order Markov
rhythms

1st -order Markov
absolute pitches

1st -order Markov
relative pitches

GR

TR

TA

TE

time

R. event

music 4 "4event

2. ANALYSIS

MIDI

rel. MIDI

Figure 3: Score analysis (Mozart cello in examples).

time

Figure 2: Audio analysis (4 measures shown in examples).

Ø

Calculatesliced means

Melodyestimation

MIDI

Chord estimation(including bass)

R. eventBeatestimation

MIDI

VQ TransformGet goodmeasuresSymbolic music

Wav file

R. event

HPSS
Harmonic portion

115

#$

& %
" "

#

" " ! !

0 2 2 2 1 3 3 0 0 2 1 3 0 0 1 1

Figure 4: Rhythmic events detected in the score.

2.1 Features needed from audio
Knowing which pitches are in the polyphonic music isuseful in creating cover songs. Since multi-pitch analysis
methods often suffer from pitch and onset detection errors
when handling polyphonic music with a drum track, wecannot simply apply the analysis beforehand and use theanalysis results as constraints. However, we can use theaudio feature extraction portions of multi-pitch analysis to
aid in generating cover songs. Concretely, after performing
Harmonic/Percussive source separation, the magnitudes andonsets of each note are obtained by applying a variable-Q
spectral transform and calculating the salience function of
the onset events.
The melody, chords, bass, and beats of a song provide
musical facets which should be observed in the cover version of a song. These facets are extracted from the audio using Songle, a music understanding engine [13]. The melody
and bass pitches, as well as the chord labels, are segmented
according to the time grid provided by the analyzed beats.
Later, these will be combined with the beat-aligned audio
spectral analysis to form probabilistic constraints.
2.2 Audio analysis
Figure 2 shows an overview of the audio analysis. We perform Harmonic/Percussive source separation with medianfiltering [9], then use a variable-Q transform (VQT) [18]with a Blackman-Harris window and the variable-Q parameter set to use a constant fraction of the equivalent
rectangular bandwidths [11], giving us spectral analysis
S. The frequency range was set to 65 Hz–2500 Hz (MIDI
pitches 36–99).
We then perform beat estimation on the original audiowith Songle and divide each beat into 4, giving 16th notes.
The means (over time) of VQT bins that fall within therange of each 16th note are calculated, producing the sliced
spectrogram AM . AM is normalized to the range [0, 1].
To estimate onset probabilities in the target song, we
use two methods: flux of AM and first-derivative SavitzkyGolay filtering [17] on S. The flux of AM is simply thehalf-wave rectified difference between successive 16th notesof AM . For the latter method, we calculate the smoothedfirst derivative of S along the time axis using SavitzkyGolay filtering with a window size of 21 samples to findthe peaks of S. To quantize the onset estimation to the16th -note level, we find the maximum peak within a time
window equal to a 16th note duration, but shifted backwards
in time by 25% to accommodate slight inaccuracies in thebeat detection. Both methods operate on each MIDI pitch
independently. We set AO to be the sum of the two methods,
and normalize it to the range [0, 1].
Finally, we extract two more pieces of information using[13]: the melody M , and the chords in each song, including
both the overall chord name C and the bass pitch CB .
2.3 Features needed from the score
Features obtained from the score analysis contribute tomaintaining the musical style. Classical string quartet music rarely includes complex rhythms and very large pitchintervals, so we obtain these tendencies as probabilities ofrhythm and pitch intervals from the corpus of scores.
2.4 Score analysis
Figure 3 shows an overview of the score analysis. We
used the Music21 [5] toolkit and corpus to analyze string
quartets by Haydn, Mozart, and Beethoven. Our analysis
comprised of pitches and rhythms, and only used music in
4/4 time which fit into a 16th -note grid. If the time signaturechanged in the middle of a movement, we only considered
the portion(s) in 4/4.
We calculated the probabilities of rhythmic events ina 16th note grid. Rhythmic events were defined as one offour possible values: 0 indicated a new note, 1 indicateda new rest, 2 indicated a continued note, and 3 indicateda continued rest; an example is shown in Figure 4. This
resulted in a 4x16 matrix of probabilities GR , with eachprobability being the number of occurrences divided by the
number of measures.
We extracted 1st -order Markovian [2] rhythm transitions.
This is simply the probability of each [previous event, next
event] pair occurring, and produced a 4x4 matrix TR .
We calculated 1st -order Markovian pitch transitions forboth absolute pitches and relative pitches. We considered achord-note or pair of chords to include every pitch transition between the notes in successive chords. For simplicity,
we recorded these transitions in two 100x100 matrices TA

116

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015
Violin 1
melodyoutput notes

Sheet music corpus

Wav file

avoid
notes
output notes⌦

output notes

Audio analysis

Corpus analysis

AO , C

V

avoid
notes

Violin 2 & Viola
shared DPM

Other
instruments

IR

EH

GR

⌦

Figure 5: Overview of creating parts.

H

E
EO

stack

time

and TE , even though a classical string quartet will not haveany notes below MIDI pitch 36. For the absolute pitches,
we added a 10 3 chance of any transition between valid
pitches; this is necessary to allow some modern pop songs
with non-classical chord progressions to be generated, particularly in the cello which is limited to the bass notes CB .
3. PROBABILISTIC GENERATION
Figure 5 gives an overview of generating the quartet parts.
First, the violin 1 part is set to the melody. Second, thecello part is generated with a probabilistic method and dynamic programming. Third, the violin 2 and viola parts are
generated together via the same probabilistic method anddynamic programming.
To prepare for the dynamic programming, we need todefine the emission and transition matrices, denoted by E
and T , respectively. Our time unit is 16th notes, and we
consider 200 possible events for each time-slice: 0 is a rest,
1–99 are note onsets of the same MIDI pitches, 100 is aheld rest, and 101–199 are held notes (of MIDI pitch +100).
We define N as the number of 16th notes in the target song.
An overview of calculating E and T is shown in Figure 6.

T

old eventFigure 6: Calculating emission and transition probabilities
E and T . ⌦ indicates element-wise multiplication.
• G0R — Rhythm grids: We take the overall probabilityof a rhythmic event in the corpus at each 16th note GR ,and repeat it for every 16 time-slices in N .
• IR — Extreme instrument ranges: We specify maximum bounds for instrument ranges: MIDI pitches 36–69
for cello, 48–81 for viola, and 55–99 for violin. Whena corpus of symbolic music is used, the pitch transition
probabilities narrow these ranges; IR is only relevant if
the user chooses not to use any corpus.
• V — Avoid previously-used notes: We reduce the probability of using the same notes as other instruments bysetting them to 10 2 in V ; other events are set to 1. We
also reduce the probability of playing a note one octave
higher than an existing note (as those are likely octave
errors in the audio analysis) by likewise setting thosevalues to 10 2 .
We eliminate any non-rest values less than 10 3 to reduce the computational load for music generation.
The transition probabilities T are a matrix of size 200⇥200,
representing every possible event-to-event pair.

3.1.1 Construction emission probabilities E
The emission probabilities E is a matrix of size N ⇥ 200,
representing every possible event at every 16th note. They
are generated by calculating EO (onsets) and EH (heldnotes), each of size N ⇥ 100,
EH = A0M ⌦ C 0 ⌦ G0R ⌦ IR ⌦ V

Create transition

3.1.2 Construction transition probabilities T

3.1 Constructing probability matrices

EO = A0O ⌦ C 0 ⌦ G0R ⌦ IR ⌦ V

TR , TA , TE

AM , C

new event

Cello
DP

event

Song information

(1)
(2)

where ⌦ is the element-wise product. The intuition behind
this multiplication is that we consider each variable to be
an independent probability distribution, so we are calculating the joint distribution. EO and EH are then stacked
vertically to form E. The variables are:
• A0O , A0M — Audio onsets and magnitudes: Audio onsets
AO and magnitudes AM for MIDI pitches 1–99 aretaken directly from the audio analysis. The “silence”
event (0) is set to a constant value of 10 5 .
• C 0 — Chord tones: We construct a matrix of all MIDI
pitches for every 16th note in the song; each cell is 1 ifthat pitch is in the given chord, 10 2 otherwise. For thecello, we use the bass note of each chord CB ; for other
instruments, we use any chord tone included in C.

T = TR0 ⌦ TA0 ⌦ TE0 ⌦ H(3)The variables are:• TR0 — Rhythm transitions: We use TR , the probabilityof each rhythm event following a previous rhythm event.
The note onset and held note probabilities are copied tovectors 1–99 and 101-199 respectively, while the rest
onset and held rest probabilities are copied into vectors
0 and 100.
• TA0 , TE0 — Pitch transitions: We use the probabilitiesof each pitch following a previous pitch considering
absolute or relative pitches, TA and TE respectively.
These matrices are originally 100⇥100; we simply copy
the matrices four times to create 200⇥200 matrices (thatis to say, allowing these relative transitions to apply toonset-onset, onset-held, held-onset, held-held pairs).
• H — Hold-events only after onset-events: Each “hold”
event (events 100 and up) can only occur after its respective “onset” event. We formalize this constraint
as a matrix H where rows 0–99 are all 1, while rows
100–199 contain two identity matrices (in columns 0–99
and 100-199).

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015
Extendeventsln(E[t])ln(T )

200
E[t]

100
0

+

100
0

0 t 1,000 2,000
time (16

th

0

!

100

200

new events

ln(E)
200

117

200
100
0
0
100 200
previous eventsnotes)

L

L[0, ·, ·]L[t, ·, ·]L[2400, ·, ·]Figure 7: Combining emission and transition probabilities
E and T into overall log-probabilities L.
3.2 Generation of a cover song under constraints
We combine E and T to form the log-probability L of thearranged score given the observed audio and corpus data,
which has dimensions N ⇥ 200 ⇥ 200. For each t 2 N ,L[t, ·, ·] =ln(E[t, , :])ln(T )(4)

where E[t, , :] indicates that the 1x200 column vector E[t]is extended to form a 200x200 matrix. This is illustrated in
Figure 7. Since E and T contain very small numbers, we
add their negative log-values instead of multiplying them.
L can be visualized by considering it to be a networkof time-events (Figure 8). The maximum probability ofa given score occurs when the negative log-probability is
minimized; i.e. by finding the shortest path through L witha standard dynamic programming algorithm [3].
3.2.1 Local and Global Shortest Paths
As shown in Figure 5, we calculate the cello accompanimentpart first. After that, we could solve the viola and thenviolin 2 parts separately, but we found that this occasionallyproduced very high violin 2 music. Instead we solve theviolin 2 and viola parts together, with the constraint thatthey cannot play the same pitch at the same time.
In order to find two shortest paths simultaneously, weconstruct a large network with every possible combinationof nodes from each time-slice of the individual violin 2 andviola networks. For example, if at time t the violin 2 could
have 4 possible events and the viola could have 5 possible
events, then the combined network will have 20 possible
events for time t. The edge weights are simply the sum of
the existing edges from the individual networks.
3.2.2 Compacting Matrices
To lower memory usage and improve processing time, we
reduce the size of the matrices. We construct a mapping
for each time-slice t between the non-infinite weights in L
and a smaller matrix. This takes approximately 1 second,
and results in a matrix which is roughly 1% of the original
size (e.g., 96 million entries reduced to 1.2 million entries).
Note that this compression is lossless and does not affectthe shortest-path calculation, as an edge with weight 1 will
not appear in the shortest path.

Figure 8: Network of possible pitches L; shortest pathcolored red. Node labels are in the form “time-event”,
with event x being a MIDI pitch onset (x < 100) or hold(x 100). For legibility, edges with a weight of infinity andnodes with no non-infinite-weight edges are not displayed.
“Compacting” L in this way speeds up the computationof the single cello part, but its true value is found when combining the violin 2 and viola parts. Without any compacting,a normal pop song (150 measures) produces a network fora single part with 2400 ⇥ 200 ⇥ 200 = 9.6 · 107 entries.
However, naively combining the violin 2 and viola parts
produces a network with 2400 ⇥ 2002 ⇥ 2002 = 3.8 · 1012
entries (15 TB of memory). We therefore perform tworounds of compacting; before and after combining the parts.
After compacting the individual violin 2 and viola parts,
we are left with networks of size approximately 1.6 million
and 2.3 million. After performing the second round of compacting (this time on the combined matrix), the memory
requirement is reduced from 5.8 GB to 0.25 GB.
3.2.3 Weighted probabilities
We found that the initial system produced music which was
too heavily biased towards one “prototypical” measure of
rhythms for each composer. We therefore multiplied eachmatrix by its own weighting factor, and allowed the user tospecify and experiment with their own desired weights.
4. EXAMPLES AND DISCUSSION
To illustrate aspects of the generated music, we created a
few cover versions of “PROLOGUE” (RWC-MDB-P-2001
No.7) from the RWC Music Database [12], with a variety
of weights to the probability distributions. Short excerptsof the beginning of “PROLOGUE” are shown in Figure 10
with four variations: no corpus analysis, no audio spectral
analysis, equal weights, and a set of custom weights.
Figures 10a and 10b clearly demonstrate the usefulnessof combining audio with score analysis. Figure 10a does
not use any corpus information (the weights of GR , TR ,TA , and TE are set to 0), and produces music which is
not idiomatic and is extremely difficult to perform. In theother extreme, Figure 10b uses the full Haydn string quartet
corpus analysis, but does not use any spectral information

Figure 10c uses all available data with weights of 1,
and the music is both quite playable and more interesting
than Figure 10b. There is more variation in the rhythms,
and most notes are typical classical-style durations suchas whole notes, half notes, or quarter notes. There area few non-chord pitch changes (e.g., violin 2 measure 3,
viola measure 13), but not many. This version contains one
mistake: the viola in measure 13 begins with a C\ 16th note
which quickly changes to a C] chord-tone. This could be
avoided by decreasing the probability of non-chord tones,
but doing so would also decrease the chance of a non-chord
tone in the original song from being reproduced. This is an
illustration of the choices available to the user.
Figures 10d (Haydn), 10e (Mozart), and 10f (Beethoven)demonstrate a custom set of weights. After some experimentation, we (subjectively) chose to set the onset AO
weight to 0.9, the corpus rhythms GR and TR weights to
0.5, and the corpus pitch transition TA and TE weights to
0.25. These three cover versions produce noticeably distinct
music, arising solely due to the corpus used. The overall
distribution of rhythmic durations seems natural: the cellohas longer notes than the inner two voices. The distributionof pitches is reasonable, with all instruments playing in a
comfortable range; the corpus clearly helps in avoiding theextreme pitches that were present in Figure 10a.
A few parts of the cover versions are the same in allcompositions. Measure 10 always ends with a G]-C\ (alternatively “spelled” as B]) in the cello and violin 2, with theviola filling in a transition from D] to C\ (or B]); this makesa nice V–I chord sequence (G] major to C] major) leading
into measure 11. In addition, the V–I resolution in measures 10–11 always includes contrary motion in the celloand violin 2. Our probabilistic generation does not takerelative motion of multiple voices into account, so this nicevoice leading must arise from the strength of its presence inthe audio spectral analysis.
A few problems exist in the voice leading. For example,
Figure 10d shows a number of parallel fifths (e.g., violaand cello, measures 4!5!6, 8!9). These likely arise
due to the 2nd and 3rd harmonics of bass guitar notes in theoriginal recording. A similar problem occurs with sudden
jumps of an octave after one 16th or 8th note appears in a few
places (e.g., viola measure 4 and cello measure 12). Thesealso likely arise due to inaccuracies in the spectral analysis:
the energy in upper partials of a single note can vary, so
multiple onsets are detected in close succession. Moreadvanced signal processing in terms of onset estimation or
pitch salience calculation could mitigate this issue. Anotherfix for the parallel fifths would be to use a more advanced
mathematical model; a first-order Markov model does nottrack the inter-dependence between quartet parts.

0

0.5

1

0.08
0.06
0.04
0.02
0

Viola Rhythms
x-corr.

Violin 2 Rhythms
x-corr.

0.08
0.06
0.04
0.02
0

0

weight of RG , T R
Violin 2 Pitches

0

0.5

1

1

0.4
0.3
0.2
0.1
0

Viola Pitches0weight of T A , T E

0.5

1

0.4
0.3
0.2
0.1
0

weight of T A , T E
Haydn

MozartCello Rhythms

0

0.5

1

weight of RG , T R

x-corr.

0.4
0.3
0.2
0.1
0

0.5

0.08
0.06
0.04
0.02
0

weight of RG , T R

x-corr.

(the weights of AO and AM are set to 0), and producesmusic which is playable but very repetitive and “boring”:
Other than measure 10 (the transition from the introduction
to the melody), each instrument in the accompaniment plays
the same rhythm in every measure (with the exception of
the cello in measure 5), and 76% of measures contain asingle pitch while 24% of measures contains two pitches.

x-corr.

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015

x-corr.

118

Cello Pitches

0

0.5

1

weight of T A , T E

Beethoven

Figure 9: Objective analysis of weights; unless otherwise
specified, our custom weights are used.
4.1 Objective analysis
Figure 9 shows the effect of changing the rhythmic orpitch corpus weights. The “pitch” plots show the crosscorrelation between the corpus relative pitch distribution
TE and the relative pitches calculated from the generated
scores. The “rhythm” plots show the cross-correlation between corpus and generated scores, based on the types ofmeasures appearing in the output. Concretely, we construct a dictionary of full-measure rhythmic events (suchas 0222133002130011 from Figure 4) along with their frequency of appearance, for both the corpus and the generated music. We then calculate the cross-correlation between
those dictionaries for the corpus and each cover version.
Increasing the weight generally increases the correlation between corpus and generated music for both pitches
and rhythms. One counter-example is violin 2 and violain Mozart quartets. We theorize that this arises because increasing the rhythmic weight reduces the number of “completely eighth note” measures in the generated music, however such measures are very common in the original corpus.
5. CONCLUSION AND FUTURE WORK
We presented Song2Quartet, a system for generating string
quartet cover versions of popular music using audio andsymbolic corpus analysis. Both the target pop song audio
file and the corpus of classical music contribute to the output; using only one or the other produces clearly inferior
results. In order to avoid awkward second violin parts, we
performed a semi-global optimization whereby we createdthe second violin and viola parts at the same time.
The current system makes a number of ad hoc assumptions, such as the melody always being played by the first
violin and all rhythms fitting into 16th -note rhythms. Ourevaluation was primarily based on informal listening, whichshowed promise despite some voice leading errors.
We plan to extend the data-driven corpus analysis so thatusers may generate cover versions for other groups of classical instruments. We also plan to add a GUI so that users
can place the melody in different instruments at any pointin the song. Finally, we would like to include evaluationsof the generated scores’ “playability” by musicians.
Acknowledgments: This work was supported in part by
CREST, JST.

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015119- )))) " = #121
#
#
#
#
#
#
+
,
"
"
"
" "
"
"
"
" "
"
"" " """" " " "" " " """""" " """ " """ " "
))))
"
"
"
"
"
"
"
"
"
"
+
"
"
"
" " " " "vln-2 ,
" "" " "" " " " " "" " " """ " " " " " " " " " " "" " " " " " " "
"
"
"
!
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
" " " " ""
"!
"" "
"
"! " " " " " "" " " " " " " ! "" "$"* )))) +$vla"" """""" " "! " " " " "" " "0 " " " "! "! " "! " " "
&
"
)
!
!
"
"
!
/
)
"
"
"
&
)+ '
$"
&
"
"
(
vlc
'
" "" ""!
"!
" % " "!
. )
"
"
" "
"
" "!
"
& !
-8 )))) #
&
&
#
#
&3
2
2!
1
,
""
" " " " " "" " !
" " " " " "
"
"
"
"
" "
"
"
"
"
" "
"
"
"" " """ " " " " 6"
"
)) "
""
"
6" " " " "" "" ", ) ) ""! " " "
" ""
""
" "
"! "
" " " " "" " "" " "5)4 " " " ""6"!
"" " " " " " " " " " " " "" " "! "
"
"
"" "
"
""
"
" $
" "
"
" "
"
"
* ))))& """" """" " "" " " "" " " " " " "" """ " " " " 0 " " "! " " " " " "! " "!"" " " " "!
("6" " " " "" " "" " "$$&" " ! $(""" " $/ ) ))&" " "!
$"$"$6""" "! "!
"!
" (". ) (!
""! " " 6""!
" " (!
"!
"
" "
"
"
vln-1

/ $$$$ ! = '121'.
vln-1 $$$$ .
& ! ! &
vln-2 -

0

1 $$$$ . &0+ + ! = 121! ! &/ ++. "++++ . # #vln-2 /
3 ++++ . vln-1

vla

vlc

1

2 ++++ . -0 //// " = %121.
//// . ! !
vln-2 vln-1

3 //// . ,

vla

vlc

'

'

'

'

'

'

'

! ! &

! ! &

! ! &

! ! &

! ! &

! ! &

! ! &

) *
) ) )+ %*,! ! ! ! ! !! ! * ! ! ! ! ! ! ! ! ! !
)! ! ! ! & ! ! & ! !
! ! ! * ( ! ! ! #$" & ! !

! ! &

! ! &

! ! !

! ! &

! ! &

! ! &

! ! (!*

"

"

'

%

2 $$$$ . ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ( ! ! !

vla

vlc

(a) “PROLOGUE” with audio analysis but no string quartet corpora.

12 //// . ,1. . ! = 121

0. . ! = 121! ! ! &)!! !

!! ! !

! ! ! !

&&

! !

! ! ! ! ! ! ! !
&

! !

! ! &

! !

(b) “PROLOGUE” with Haydn string quartets but no audio spectral analysis.
"
# #
# #

"

! $ !% #
#$
#$

"
#

#

"

"

# #

#$

! ! ! ! !
$
#
!
!
-

"

"

(

% $
*
! ! ! ! ! !! ! $ ! ! ! ! !

%%# ! ! # ! ! ! ! '# ! ##!
$!
# ! ! ! ! ! ! # #
!
! !
!
&
!
!
# #
!
$
%
#
!
$
'
!
#
#$ ! # #
#$
# ! #
! -

#$

!

! #
!

%!$ #

!

% %
$
! ! ! ! !% ) (! ! ! #%
' ! +! ! $ ! ! # $
#
#
#
!

#

,

!
! !

(c) “PROLOGUE” with Haydn string quartets and all weights set to 1.0.
%

%

! !

%

%

%

%

%

%

(

# $
# #
*
" " " " " "" " $ " " " " " " " " ""$ &" " ""$ #" ! " #"'
" "$ " " ! " " " """ " " " " " " ! """!
! "" ! " " ## " " &"
"
"
"
"
" " "
!
"
!
"
!
,
+" "$ " "! "$ "
&"
$
#
!
$
&
"
!$
"
!
!$ " ! ! !$
" ! !
! " !
" ,

! " " " " "# !
'
" !$

%

(d) “PROLOGUE” with Haydn string quartets and custom weights.

"

#) ($

#
" " "$ " " " " " " " " " !
" ""' "$ " ' "$ " " !$"##" " !
! !
! !
"

% "
% %
&
&
&
&
&
&
&
&
&
%) $ "*$
0 ..- &
! ! ! ! ! !! ! " ! ! ! ! ! ! ! ! ! !
".... - ! " ! ! " ! ! ! ! ! ! ! !
!
%'!! ! ! !
%%%vln-2 0#
! ! ! ! ! ! ! ! ! ! ! ! !" ! ! ! !" ! ! ! ! ! ! ! ! # ! ! ! ! !
!
( (
(
!
!
"
!
!
!
!
!" ! ! ! ! ! ! ! ! ! ! ! ! ! !
! "! ! ! ! # !! " !
,!
%'!
/ .... - !
%
%
!
vla
!
! ! ! ! ! ! ! ! ! ! ! ! # !"! # ! ! ! ! ! ! ! ! ! ! # #! '! ! ! ! ! ! ! ! ! ! ! " ! ( ! " ! ! ! ! " ! " !
!" ! !" ! !
%%# ! '! !
3 . .. - +%! ! #! ! # ! +vlc
# # #"! !! #
#"
#"
! !
! # #
2 .
# # # #(#!
vln-1

/ ..- &.... - # #vln-2 /
vln-1(e) “PROLOGUE” with Mozart string quartets and custom weights.

&

&

&

&

# ! ! ! ! !" # !
# #
'
! ! !
#
!
#
%
!
.
.
! # ! ! "3 ..- !
vla*,! !% #*
2 .... - *
#
%
!
vlc
#
#
*
1

&

&

! # !

&

! ! ! ! ! ! !

# ! ! ! !!! ! #
#%

! *

#%

&

&

$

" %
" "
+
! ! ! ! ! !! ! % ! ! ! ! ! ! ! ! !

(!
"!
! # ! ! ! ! ! (!%
! #
!
%
!
! ! !
!
' ! (!
## #! #% ! (!%
"#
#
# !

(f) “PROLOGUE” with Beethoven string quartets and custom weights.

Figure 10: Sample output; full versions and synthesized audio available at:https://staff.aist.go.jp/m.goto/Song2Quartet/

! #
!
#
!

"!% #

!
!

!

")

$%

! ! ! !! ! ! #! ",! !% ! ! *
#
#
# #

120

Proceedings of the 16th ISMIR Conference, Málaga, Spain, October 26-30, 2015
6. REFERENCES[1] Moray Allan and Christopher K. I. Williams. Harmonising chorales by probabilistic inference. In NIPS, pages
25–32, 2005.
[2] Charles Ames. The markov process as a compositionalmodel: a survey and tutorial. Leonardo, pages 175–187,
1989.

[14] Gen Hori, Hirokazu Kameoka, and Shigeki Sagayama.
Input-Output HMM Applied to Automatic Arrangement for Guitars. Journal of Information Processing,21(2):264–271, 2013.
[15] Francois Pachet. The continuator: Musical interaction
with style. Journal of New Music Research, 32(3):333–
341, 2003.

[3] Richard Bellman. Dynamic Programming. Princeton
University Press, Princeton, NJ, USA, 1 edition, 1957.

[16] Francois Pachet and Pierre Roy. Musical harmonization
with constraints: A survey. Constraints, 01/2001(6):7–
19, 2001.

[4] David Cope. Computer modeling of musical intelligence in EMI. Computer Music Journal, pages 69–83,
1992.

[17] Abraham. Savitzky and M. J. E. Golay. Smoothing anddifferentiation of data by simplified least squares procedures. Analytical Chemistry, 36(8):1627–1639, 1964.

[5] Michael Scott Cuthbert, Christopher Ariza, and Lisa
Friedland. Feature extraction and machine learning on
symbolic music using the music21 toolkit. In ISMIR,
pages 387–392, 2011.

[18] Christian Schörkhuber, Anssi Klapuri, Nicki Holighaus,and Monika Dörfler. A matlab toolbox for efficientperfect reconstruction time-frequency transforms with
log-frequency resolution. In Audio Engineering Society
Conference: 53rd International Conference: Semantic
Audio. Audio Engineering Society, 2014.

[6] Arne Eigenfeldt and Philippe Pasquier. Considering vertical and horizontal context in corpus-based generativeelectronic dance music. In Proceedings of the FourthInternational Conference on Computational Creativity,
volume 72, 2013.
[7] Benjamin Evans, Satoru Fukayama, Masataka Goto,
Nagisa Munekata, and Tetsuo Ono. AutoChorusCreator:
Four-Part Chorus Generator with Musical Feature Control, Using Search Spaces Constructed from Rules ofMusic Theory. In International Computer Music Conference, 2014.
[8] Jose D Fernández and Francisco Vico. Ai methods in algorithmic composition: A comprehensive survey. Journal of Artificial Intelligence Research, pages 513–582,
2013.
[9] Derry Fitzgerald. Harmonic/Percussive Separation using Median Filtering. In International Conference on
Digital Audio Effects, 2010.
[10] Satoru Fukayama and Masataka Goto. Chord-sequencefactory: A chord arrangement system modifying factorized chord sequence probabilities. In ISMIR, pages
457–462, 2013.
[11] Brian R Glasberg and Brian CJ Moore. Derivation ofauditory filter shapes from notched-noise data. Hearing
research, 47(1):103–138, 1990.
[12] Masataka Goto, Hiroki Hashiguchi, Takuichi
Nishimura, and Ryuichi Oka. RWC Music Database:
Popular, Classical and Jazz Music Databases. In ISMIR,
volume 2, pages 287–288, 2002.
[13] Masataka Goto, Kazuyoshi Yoshii, Hiromasa Fujihara,
Matthias Mauch, and Tomoyasu Nakano. Songle: A
web service for active music listening improved by user
contributions. In ISMIR, pages 311–316, 2011.