AUTOMATIC GENERATION OF METRICAL STRUCTURE
BASED ON GTTM
Masatoshi Hamanaka
PRESTO, Japan Science andTechnology Agency
m.hamanaka@aist.go.jp

Keiji Hirata
NTT Communication Science
Laboratories
hirata@brl.ntt.co.jp

ABSTRACTThis paper describes an automatic system of music
analysis based on the generative theory of tonal music(GTTM). The system works by acquiring the metricalstructure of the music. The GTTM is considered one of
the most promising theories of music in regard tocomputer implementation; however, no order is given
for applying its rules and as a result, they frequently
conflict with each other. To solve this problem, we
introduced adjustable parameters, enabling us to assign
priority to the rules. Our GTTM-based system makes it
possible to construct hierarchical metrical structures in a
top-down process using bottom-up detection of local
metrical strength. Experimental results showed that after
these parameters were tuned, our method outperformed
the baseline performance.
1.

INTRODUCTION

We are developing a system of music analysis based on
the generative theory of tonal music (GTTM) [1]. The
GTTM is composed of four modules, each of whichassigns a separate structural description to a listener’s
understanding of a piece of music. These four modules
output a grouping structure, metrical structure, timespan tree, and prolongational tree, respectively. Our
goal is to automatically derive a time-span tree thatassigns a hierarchy of ‘structural importance’ to thenotes of a piece of music (Figure 1). Automatic
generation of a time-span tree from the music surface
enables us to analyse the deeper structure [2]. It will
also provide music summarization [3] and collaborative
music creation [4] systems that will enable users tomanipulate music by using a time-span tree,
disregarding the surface structure of the music.
To derive a time-span tree, we use the grouping andmetrical structures of the music. In previous work [5],we developed a system for automatically generating agrouping structure. In this paper, we describe a method
for automatically articulating the transition of notes intoa metrical structure.
Previous methods based on beat tracking [6, 7] areonly able to acquire the hierarchical metrical structure ina measure because they do not consider larger metricalstructure such as two measures, four measures, and soon. We have previously attempted to implement several
rules of the GTTM, but when these rules conflicted wewere unable to adequately resolve the priority of multiple

Satoshi Tojo
Japan Advanced Institute of
Science and Technology
tojo@jaist.ac.jp

rules [8, 9]. Although a computer model of GTTM [10]is capable of producing a time-span tree, we, human,need to choose applicable rules at each stage ofprocessing.
Our system of metrical analysis based on the GTTM
makes it possible to construct a hierarchical metricalstructure automatically by iterating calculation of lowlevel beat strength and choosing the next level structure.
Time-span tree…
Metrical structure・・ ・ ・・ ・ ・・ ・ ・・ ・ ・・ ・ ・・ ・ ・・ ・ ・・ ・
・
・
・・
・・・

Grouping structure

Figure 1. Time-span tree, metrical structure, andgrouping structure.

2.

METRICAL STRUCTURE ANALYZER
BASED ON GTTM

The metrical structure describes the rhythmicalhierarchy of a piece of music by identifying the positionof strong beats at the levels of a quarter note, half note,a measure, two measures, four measures, and so on.
Strong beats are illustrated as several levels of dots
below the music staff (Figure 1).
Figure 2 shows the processing flow of the system. As
the primary input formats we chose MusicXML [11]and GroupingXML [5]. A hierarchical metrical structure
was constructed in the top-down way, while the lowerlevel beat strength are detected in the bottom-up way.
We then designed MetricalXML as the export format
GroupingXML

MusicXML
Calculation of low- Current structurelevel beat strength
low-levelDi(strength of beat)Choosing nextlevel structure

Applying MPR1,2,3,4,5 [time]mˆ = 1

Choice of next mˆ = 2
ˆ =3
level structures mmˆ = 4mˆ = 5Choosing with applying MPR10

Contains more than one beatNo

Yes

MetricalXML

Figure 2. Processing flow of metrical structure analyzer.

for our system; note elements in MetricalXML areconnected to note elements in MusicXML andGroupingXML by Xlink [12] and Xpointer [13] (Figure3). In our experiments, we restricted the music structure
to monophony so we could more accurately assess theperformance of each rule.
Xlink
Grouping-XML
Xpointer - <group>- <group>MusicXML- <part id="P1">- <measure number="1">+ <note>+ <note>+ <note>+ <note></measure >- <measure number=“2">+ <note>+ <note>+ <note>+ <note>+ <note></measure >- <measure number=“3">+ <note>::+ <note id="P1-1-1">+ <note id="P1-1-2">+ <note id="P1-1-3">+ <note id="P1-1-4">+ <note id="P1-2-1"></group><applied rule="2a" /><applied rule="6" />- <group>+ <note id="P1-2-3">+ <note id="P1-2-4">+ <note id="P1-2-5">+ <note id="P1-2-6">+ <note id="P1-3-1"></group></group>::<applied level=0.50 rule=3><applied level=1.00 rule=3>+ <note></metric>- <metric dot="3" at= “0.25" ><applied level=0.25 rule=3>+ <note></metric>- <metric dot="3" at= “0.50" ><applied level=0.25 rule=3><applied level=0.50 rule=3>+ <note></metric>+<metric>+<metric>::

2.1. Application of MPRs
There are two types of rules in the GTTM, i.e., “wellformedness” and “preference” rules. Well-formedness
rules are necessary conditions for the assignment of astructure and restrictions on the structure. When morethan one structure satisfies the well-formedness rules,
the preference rules indicate the superiority of one
structure over another.
The metrical preference rules (MPRs) consist of 10
rules: MPR1 (parallelism), MPR2 (strong beat early),
MPR3 (event), MPR4 (stress), MPR5 (length), MPR6
(bass), MPR7 (cadence), MPR8 (suspension), MPR9(time-span interaction), and MPR10 (binary regularity).
MPR5 has six cases: (a) pitch-event, (b) dynamics, (c)slur, (d) articulation, (e) repeated pitches, and (f) harmony.
Here we discuss the application of MPR1, 2, 3, 4, 5,
and 10. The strength of the beat dependent on each rulecan be expressed as DiMPR j (=1,2,3,4,5a,5b,5c,5d,5e) (0≦DiMPR j≦1). Because there is no strict order for applying MPR
rules, ambiguities in the analysis may result. To solve
this problem, we use adjustable parameters SMPR j(=1,2,3,4,5a,5b,5c,5d,5e,10)that enable us to control the strengthof each rule. Figure 4 indicates the relationship between
the parameters and MPRs. Our metrical structure
analyzer has sixteen adjustable parameters, which
include SMPR j, Wr, and TMPR j (Table 1).

istart
iend

(1)

Wr

DiMPR2

MusicXML(3)

DiMPR3(4) ～(7)

veloi
valui
voli
sluri
numi

(10)

SMPR jj =2,3,4,5a,5b,5c,5d,5e(2)

TMPR jj =4,5a,5b,5c

(8)
(9)

DiMPR4
DiMPR5a
DiMPR5b
DiMPR5c
DiMPR5d
DiMPR5e

TMPR jTable 1. Sixteen adjustable parameters.

2.1.1. Calculation of basic parameters
As calculated from MusicXML, the five basic
parameters of a note from beat i are: veloi (velocity),
valui (length of note), voli (duration of dynamic1), sluri(length of slur), and numi (pitch). µvelo, µvalu, µvol, µslur,and µnum are the average of the basic parameters. As
calculated from GroupingXML, the two basic
parameters i start and i end are the beginning and ending ofa group that is the smallest group containing i and morethan one beat in the current structure.
2.1.2. Application of MPR1
MPR1, which is the rule for parallelism in a metricalstructure, has two adjustable parameters: Wr (weight ofpriority of the same rhythm compared with the sameregister in parallel segments), and TMPR1 (threshold that
decides whether beat i and beat k are parallel, i.e.,(Di kMPR1=1) or not (Di kMPR1=0)).
⎧⎪1
DiMPR1
=⎨
k
⎪⎩ 0

yi k
z
× Wr + i k × (1 − Wr ) > T MPR1
xi k
yi k
else
,

(11)
Dilow-level

^
m

Bi
SMPR1

(1)

where
k end ⎧1 velo ′ > 0
i end ⎧1 velo ′ > 0kixi k = ∑ ⎨
+ ∑ ⎨
0
velo
0
0
velo
=
start
start
′
′ =0
k
i
⎩
⎩
′
′
k =k
i =i
i end ⎧1 velo > 0 and veloik +i′+i > 0
yi k = ∑ ⎨
e
lse
0
start
⎩
i′=i

zi k =

i end⎧1 veloi > 0 and numi′-1 = numi′ and numk +i′−i -1 = numk +i′−i0 else.

∑ ⎨i′=i start ⎩2.1.3. Application of MPR2 and 3
MPR2 has a weak preference for a metrical structure inwhich the strongest beat in a group appears relatively
early in the group. We formalized DiMPR2 so that thecloser it is to the beginning of the group, the higher thevalues. MPR3 prefers a metrical structure in which the
inceptions of pitch-events are strong beats (Figure5).
(2)
DiMPR2 = i end − i i end − i start(

Di kMPR1
TMPR1

Description
Strength of each rule.
j =(1, 2, 3, 4, 5a, 5b, 5c, 5d, 5e, 10)
Weight of priority of the same rhythm comparedwith the same register in parallel segments.
Value of the threshold that decides whether or noteach rule is applicable. j =(1, 4, 5a, 5b, 5c)

Wr

Xlink
MetricalXML
dot="3" at= “0.0" >Xpointer - <metric<applied level=0.25 rule=3>

Figure 3. MusicXML, GroupingXML, and Metrical XML.

GroupingXML

Parameters
SMPR j)()

veloi > 0

⎧1
DiMPR3 = ⎨⎩0(3)veloi = 0…

SMPR10Grouping structure

DiMPR2[i]0(n) :formula

DiMPR3variable0[i]Figure 5. Application of MPR2 and 3.

parameter
1

Figure 4. Relationship between parameters and MPRs.

Metrical structure(Current structure)

The GTTM does not define the meaning of ‘duration of dynamic’.
Therefore, we define it as the length from one beat to the next beat orrest, depending on the example of applying MPR5b in the GTTM [1].

2.1.4. Application of MPR4, 5a, 5b, and 5c
MPR4, 5a, 5b, and 5c prefer a metrical structure in whicha relatively strong beat occurs at the inception of either astressed note, a long note, a long duration of a dynamic,
or a long slur. We introduced the adjustable parameters
TMPR j(=4, 5a, 5b, and 5c) (0≦ TMPR j ≦1) to control the value ofthe threshold that decides whether each rule is applicable,i.e., (Di MPRj=1) or not (Di MPRj=0) (Figure 6).
DiMPR4⎧⎪1=⎨⎪⎩0valui > 2 × µ valu × T
else

⎧⎪ 1
DiMPR5b = ⎨voli > 2 × µ vol × T MPR 5b
else

⎪⎩0
⎧⎪1
DiMPR5c = ⎨⎪⎩0sluri > 2 × µ slur × T
else⎧Dilow-level metrical⎪mˆ = argmax ∑ ⎨ Dilow-level metrical × S MPR10m( =1, 2,3,4,5) i ⎪0⎩(i - m) mod 2 = 0(i - m) mod 3 = 1 (11)else…
Current structure

veloi > 2 × µ velo × T MPR 4
else

⎧⎪1
DiMPR5a = ⎨⎪⎩0structure (Figure 7). When the current structure contains
more than one beat, the next level structure m̂ iscalculated as follows:(4)

MPR 5a(5)

Dilow-level[i]

Choice of next level structure

^
m=1

(6)

[i]

^
m=2

[i]

^
m=3

MPR 5c

[i]

^
m=4

(7)

[i]

^
m=5

[i]

…
^^
5b5b
5c
velo
2μvelo

^
4
5a
5b

^
4
5a
5b

^^
5b5b
5c

^
4
5a
5b

^
4
5a
5b

^^
5b5b
5c

Figure 7. Selecting the next level structure.
Current structure

3.
TMPR4
[i]

valu
2μvaluTMPR5a
[i]

vol
2μvol

TMPR5b[i]slur
2μslur

TMPR5c[i]

GTTM EDITOR

We developed a GTTM editor to construct grouping andmetrical structures, and time-span trees. The GTTM
editor has two modes as follows.
3.1. Automatic-analysis mode
The automatic-analysis mode displays the results of ourgrouping and metrical structure analyzer (Figure 8). The
structure changes depending on the parameters configured.

Figure 6. Application of MPR4, 5a, 5b and 5c.

2.1.5. Application of MPR5d and 5e
MPR5d is the rule for repetition of an articulation
pattern. We apply MPR5d to those positions where
there are inceptions of MPR5a repetition. MPR5e is therule for pitch-repetition.
⎧⎪1
DiMPR5d = ⎨⎪⎩ 0⎧1
DiMPR5e = ⎨⎩0=1
DiMPR5a = 1 and DiMPR5a
+1
else

(8)

numi = numi +1

(9)

else

Figure 8. GTTM editor (automatic-analysis mode).

2.2. Calculation of low-level beat strength
Low-level beat strength is calculated by weighted
summation of DiMPR j (=1,2,3,4,5a,5b,5c,5d,5e).
⎧⎪ B × S MPR1
Dilow-level metrical = Bi + ∑ ⎨ kk ⎪⎩ 0= 1 (10)
DiMPR1k=0,
DiMPR1
k

where
Bi =

MPR j
× S MPR j .
∑ Di
j = ( 2,3, 4,5a ,5b,5c ,5d ,5e )2.3. Acquisition of hierarchical metrical structure
A hierarchical metrical structure is constructed by
iterating calculation of the low-level strength of the beatfor the current structure and choosing the next level

3.2. Manual-edit mode
The manual-edit mode assists in manual editing of thegrouping structure, metrical structure, and time-span tree
(Figure 9). It can be used to edit the results of our grouping
and metrical structure analyzer in automatic-analysis mode.
Now editing thistime-span tree
Time-span tree
notes
Grouping structure
Metrical structure
Figure 9. GTTM editor (manual-edit mode).

4.

EXPERIMENTAL RESULTS

6.

We evaluated the performance of our metrical structure
analyzer using an F-measure, which is given by theweighted harmonic mean of Precision P (the proportionof selected metrical dots that are correct) and Recall R(the proportion of correct metrical dots that wereidentified). We did not take care that low-level error is
propagated up to higher-level; we counted wrong
answers in each metrical level.
Fmeasure = 2 ×

P×R
P+R

(12)

This evaluation required us to prepare accurate data
for the metrical structure. We collected one hundred
pieces of 8-bar length, monophonic, classical music, andasked people with expertise in musicology to analyze
them manually with faithful regard to the MPRs using
the GTTM editor. These manually produced results werecross-checked by three other experts.
To evaluate the baseline performance of our system,
we used the following default parameters: SMPR j=0.5,
Wr=0.5, and TMPR j=0.5. The average F-measure for thebaseline performance was 84%. We took about 10
minutes in average for finding a plausible tuning of theparameter set by hand. As a result of configuring theparameters, the F-measure of our metrical structure
analyzer reached 90%, outperforming the baseline by
6% (Table 2).
Baseline
performance

Melody1. Spinnerlied
2. Petit Chien
3. Solvajg’s Song
4. L’Arlesienne
5. Tarantella
6. The Moldau
7. Tristesse
8. Waves of the Danube
9. Barcarolle
10. Gymnopedie::
Total (100 melodies)

0.88
0.86
0.86
0.92
0.63
0.73
0.88
0.72
0.72
0.66
:
:
0.84

Our methodwith configured
parameters
1.00
0.92
0.86
0.98
0.69
1.00
1.00
0.97
0.79
0.82
:
:
0.90

Table 2. F-measure for our method

5.

CONCLUSION

We developed a system for analyzing the metricalstructure of music based on the GTTM. The system
makes it possible to construct a hierarchical metricalstructure. Our experimental result showed that the Fmeasure reached 90%, that outperforms the baseline
measure by 6%, with optimally configured sets of theparameters. At present, time-span trees are generated
manually using only MPRs together with groupingstructure information provided by GroupingXML. We
are now planning to improve the precision of the trees,
implementing further details of the time-span tree
generation rule.

REFERENCES[1] Lerdahl, F., and R. Jackendoff. A Generative
Theory of Tonal Music. MIT Press, Cambridge,
Massachusetts, 1983.
[2] Hirata, K., and T. Aoyagi. ''Computational
Music Representation on the Generative
Theory of Tonal Music and the Deductive
Object-Oriented Database'', Computer Music
Journal 27(3), 73–89. 2003.
[3] Hirata, K., and S. Matsuda. ''Interactive Music
Summarization based on Generative Theory ofTonal Music'', Journal of New Music Research,
32:2, 165-177. 2003.
[4] Hirata, K., S. Matsuda, K. Kaji, and K. Nagao.
''Annotated Music for Retrieval, Reproduction,and Sharing'', Proceedings of the International
Computer Music Conference, Miami, USA,2004.
[5] Hamanaka, M., K. Hirata, and S. Tojo.
''Automatic Generation of Grouping Structure
based on the GTTM'', Proceedings of theInternational Computer Music Conference,
Miami, USA, 2004.
[6] Rosenthal, D. ''Emulation of human rhythm
perception'', Computer Music Journal 16(1),64–76. 1992.
[7] Goto, M. ''An Audio-based Real-time Beat
Tracking System for Music With or Without
Drum-sounds'', Journal of New Music
Research, 30:2, 159-171. 2001.
[8] Ida, K., K. Hirata, and T. Satoshi. ''The
Attempt of the Automatic Analysis of theGrouping Structure and Metrical Structure
based on GTTM'', SIG Technical Report2001(42):49-54, Japan, 2001, (in Japanese).
[9] Touyou, T., K. Hirata, and T. Satoshi.
''Improvement of Grouping Rule Application in
Implementing GTTM'', SIG Technical Report
2002(47):121-126, Japan, 2002, (in Japanese).
[10] Nord, T. Toward Theoretical Verification:
Developing A Computer Model of Lerdahl andJackendoff's Generative Theory of Tonal Music,
Dissertation, The University of Wisconsin,
Madison, 1992.
[11] Recordare LLC. ''MusicXML 1.0 Tutorial''
http://www.recordare.com/xml/musicxmltutorial.pdf., 2004.
[12] W3C. ''XML Linking Language (XLink)
Version 1.0. '' http://www.w3.org/TR/xlink/,
2001.
[13] W3C. ''XML Pointer Language (XPointer). ''http://www.w3.org/TR/xptr/, 2002.