Proceedings ICMC|SMC|2014

14-20 September 2014, Athens, Greece

Algebraic Mozart by Tree Synthesis
Keiji Hirata
Future University Hakodate
hirata@fun.ac.jp

Satoshi Tojo
JAIST
tojo@jaist.ac.jp

Masatoshi Hamanaka
Kyoto University
masatosh@kuhp.kyoto-u.ac.jp

ists between our experience of music and therationalizations we use to specify it.

ABSTRACT
Thus far, we have been automatizing the time-span analysis of Jackendoff and Lehrdahl’s Generative Theory ofTonal Music (GTTM). We have also introduced the distance between two time-span trees and verified by an experiment that the distance was properly supported by thepsychological similarity. In this paper, we synthesize a
new piece of music using the algebraic operations on timespan trees, with this notion of distance. For this process,we need an operation to retain a certain number of pitch
events as well as reduction, then we employ join operation on two input pieces of music. But, the result of thejoin operation is not obvious as two or more pitch events
may occupy the same position on a score in a conflicting
way. Therefore, in this research, we distinguish the tree
representation from actual music written on a score anddefine join and meet in the domain of the tree representation in the algebraic manner. Then, to demonstrate thevalidity of our approach, we compose artificial variationsof K.265/300e by Wolfgang Amadeus Mozart by a morphing technique using join and meet. We examine the resultswith human intuitive similarity and show that algebraic operations such as join and meet suffices to produce viable
Mozartoid variations.
1. INTRODUCTION
The main aim of conventional music theories is analyzing
and understanding music, not composing. Although there
have been various attempts at applying conventional music
theories to composition [7], Roads pointed out the difficulty in these attempts as follows [9, p.909]:
The surface of any music can be encoded
into such rules. But no one would mistakethe logic of a style template as anything resembling the actual process of human composition. · · · Emotional involvement is inseparable from musical behavior of all kinds, yetthere have been only a few attempts to consider affect as part of a model of compositional thought · · · A model that relates musicalstructure to its emotional significance, however crude, may lessen the disparity that exc
Copyright: 2014
Keiji Hirata et al. This is an open-access article distributed

We have been investigating the algebraic framework for
manipulating music pieces under the principle that reduction corresponds to the partial order. Among music theories that have been proposed so far, we think that the timespan tree introduced by Lerdahl and Jackendoff’s Generative Theory of Tonal Music (GTTM; hereafter) [5] is suitable for the domain in which we formalize reduction. Let
us consider the time-span tree and reduction. The timespan analysis in GTTM assigns structural importance toeach pitch events, derived by the grouping analysis, in whicha sequence of notes forms a short phrase called a group,and by the metrical analysis, where strong and weak beats
are properly assigned to each pitch event. As neighboring notes can be compared by this structural importance inthe bottom-up way, the hierarchy forms a time-span tree,
where a branch from a less important event is absorbed
into that from a more important event. We illustrate thisprocess in Fig. 1. This theory, therefore, includes the reduction hypothesis; in the sequence of reduction of pitch
events, the original piece is simplified and is abstracted,
and thus, we can retrieve a basic skeleton [6] of the original music piece 1 .
Thus far, we have automatized the process of time-span
analysis [1], and proposed various applications [2]. In [11],we defined a notion of distance among the time-span trees,
and then we compared the tree distance with human cognitive similarity, among 12 variations of Ah vous dirai-je,
maman, K. 265/300e by Wolfgang Amadeus Mozart [4].
In this paper, we propose a technique for creating a music piece based on our algebraic framework which is bothmathematically and cognitively well-grounded. As an application of the technique, we demonstrate the compositionof new variations from two existing variations, combining
the two time-span trees of the variations in the algebraicmanner with the join and meet operations. For meet as an
operation to reduce uncommon pitch events, the meet operation is rather naturally defined as the intersection partof two music pieces. Thus, if we restrict our interest in
the calculation of distance, meet may sufficiently serve as
an edit distance such as earth mover’s distance (EMD) or
Rizo-Valero’s [8]. For join as an operation to increase pitch
events, in contrast, it is problematic because the join operation does not always function, when the two music scorescontain unmatched pitch events. Here, our idea is to intro-

under the terms of the Creative Commons Attribution 3.0 Unported License, which
permits unrestricted use, distribution, and reproduction in any medium, provided
the original author and source are credited.

- 991 -

1 Although a pitch event means a single note or a chord, in this paper,
we restrict our interest to monophonic analysis as the method of chord
recognition is not included in the original theory.

Proceedings ICMC|SMC|2014

Ordering of
reduction
proceeded

14-20 September 2014, Athens, Greece

tures. σ2 subsumes σ1 , that is, σ1  σ2 if and only if forany branch in σ1 there is a corresponding branch in σ2 .
Let σA and σB be tree structures for two music pieces A
and B, respectively.

6

join If we can fix the least upper bound of σA and σB ,that is, the least y such that σA  y and σB  yis unique, we call such y the join of σA and σB ,denoted as σA  σB .
meet If we can fix the greatest lower bound of σA and σB ,that is, the greatest x such that x  σA and x  σBis unique, we call such x the meet of σA and σB ,denoted as σA  σB .

Surface structure
Reduction

We can define σA  σB and σA  σB by recursive functions. Thus, the partially ordered set of time-span trees
becomes a lattice, where σA  x = σA and σA  x = x if
x  σA . Moreover, if σA  σB , x  σA  x  σB andx  σA  x  σB for any x.
2.2 Maximal Time-Span and Reduction Distance?

Figure 1. Reduction hierarchy of chorale ‘O Haupt voll
Blut und Wunden’ in St. Matthew’s Passion by J. S. Bach[5, p.115]
duce an algebraic domain in which a virtual representationof a join-ed time-span tree is allowed.
This paper is organized as follows. In Section 2, we provide basic algebraic operations for time-span trees, and thenotion of distance, as a background theory. In Section
3, we propose a new notion of abstract join by which wewould represent a virtual tree, and clarify the morphing algorithm in Section 4. Next in Section 5, we actually show
new variations generated by our method, and evaluate thepieces from a human psychological viewpoint. Finally, in
Section 6, we mention the limitations of our method, anddiscuss the possibility of further development.
2. JOIN REVISITED AS A SYNTHESISOPERATION
To provide a prerequisite for the following sections, we
explain our approach, excerpting necessary definitions andproperties from our previous works [4, 11].
2.1 Subsumption, join, and meet
Hereafter, we identify the reduction in trees with the subsumption relation, which is the most fundamental relationin knowledge representation. Let σ1 and σ2 be tree struc-- 992 -The head pitch event of a tree is the most salient event in
the tree; i.e., the saliency is extended to the whole tree.
As the situation is the same in each subtree, we consider
that each pitch event has its maximal length of saliency,
called maximal time-span. We hypothesize that if a branchwith a single pitch event is reduced, the amount of information corresponding to the length of its maximal time-span
is lost.
In Fig. 2 (a), there are four contiguous pitch events, e1,e2, e3, and e4; each has its own temporal span (duration on
surface), s1, s2, s3, and s4, denoted by thin lines. Fig. 2 (b)depicts time-span trees and corresponding maximal timespan hierarchies, denoted thick gray lines. The relationships between spans in (a) and maximal time-spans in (b)are as follows. At the lowest level in the hierarchy, thelength of a span is equal to that of a maximal time-span;
mt2 = s2, mt3 = s3. At the higher levels, mt1 = s1 + mt2,
and mt4 = mt1 + mt3 + s4 = s1 + s2 + s3 + s4. That is, every span extends itself by concatenating the span at a lower
level along the configuration of a time-span tree. When all
subordinate spans are concatenated up into a span, the spanreaches the maximal time-span.
The distance d of two time-span trees such that σA 
σB in a reduction path is defined byd (σA , σB ) = e∈ς(σB )\ς(σA ) se .
For example in Fig. 2, the distance between σ1 and σ4
becomes mt1 + mt2 + mt3. Note that if e3 is first reduced
and e2 is subsequently reduced, the distance is the same.
Although the distance appears at a glance to be a simple
summation of maximal time-spans, there is a latent orderin the addition, for the reducible branches are different ineach reduction step. In order to give a constructive procedure to this summation, we introduce the notion of totalsum of maximal time-spans as:tmts(σ) = e∈ς(σ) se .
When σA  σB , d (σA , σB ) = tmts(σB ) − tmts(σA ).
As a special case of the above, d (⊥, σ) = tmts(σ).

Proceedings ICMC|SMC|2014

14-20 September 2014, Athens, Greece

σϭ

Ğϭ

Ğϯ

Ğϰ

R

R

q

Ɛϭ ƐϮ Ɛϯ Ɛϰ

ŵƚϭ

R

ĞϮ

R

q

q

ŵƚϮ ŵƚϯ

D 6HTXHQFHRISLWFKHYHQWV
DQGWKHLUVSDQV

σϮ

q

q
ŵƚϰ

ŵƚϭ

q
ŵƚϯ

σϯ

q

q
ŵƚϰ

ŵƚϭ

σϰ

q

q
ŵƚϰ

ŵƚϰ

(b) Reduction proceeds by removing reducible maximal time-spans

Figure 2. Reduction of time-span tree and maximal time-span hierarchy; thick gray lines denote maximal time-spans while
thin ones denote pitch durations.
2.3 Requirement on Distance

MRLQ

As there is a reduction path between σA σB and σA σB ,and σA  σB  σA  σB , d (σA  σB , σA  σB ) is unique.
Here let us define two distance metrics.
d (σA , σB ) ≡ d (σA  σB , σA ) + d (σA  σB , σB )d (σA , σB ) ≡ d (σA , σA  σB ) + d (σB , σA  σB )UHGXFWLRQ PHHW

'ddDŶĂůǇƐŝƐ
7LPH6SDQ7UHH

0XVLF6FRUH
ZĞŶĚĞƌŝŶŐ$OJHEUDLFGRPDLQ

We immediately obtain d (σA , σB ) = d (σA , σB ) by theuniqueness of reduction distance.
Hereafter, we omit {, } from d{,} , simply expressing it as ‘d’. Here, d(σA , σB ) is unique among the shortest
paths between σA and σB . Finally, we obtain

0XVLFGRPDLQ

Figure 3. The Proposed Framework For Music Synthesis
Absorption Law (σA  σB )  σA = σA and (σA  σB ) 
σA = σ A .
Parallelism of distance d (σA , σB ) = d (σA , σB )d(σA , σB ) + d(σB , σC ) ≥ d(σA , σC ),which is the triangle inequality. For more details on thetheoretical background, see [11].

We can easily confirm that these two conditions ensure theuniqueness of join.
3. REPRESENTATION OF TIME-SPAN TREE

2.4 Framework for Music Synthesis
To synthesize a new piece of music, one may plan to use
meet to reduce and join to increase the number of pitch
events from two concrete music scores. In actual fact, meetmostly works well, while the result of join is, however, often not obvious as two or more pitch events may occupy
the same position on a score in a conflicting way. Therefore we propose to provide a virtual join representation, notfor concrete music score, but for time-span trees, to apply
it to the morphing, as described in the following section.
Here, we state that the time-span tree representation
should be strictly distinguished from the actual music represented on scores (Figure 3). The left-hand image in Figure 3 refers to the algebraic domain which we mentioned
in preceding subsections. On the contrary, the right-handside of the figure refers to the domain of actual music. To
go from a tree representation to a concrete music score,
we need another process of music rendering, which is independent of the process of analysis from music scores totrees [1]. At the same time, however, this implies that we
do not need to concern ourselves with the actual image ofmusic in these algebraic operations.
Instead of an algebraic lattice where meet and join exist uniquely, we need to specify the requirements for thetree representation of join; we should summarize this asfollows:- 993 -In this and the following sections, we present new contributions of the paper.
Thus far, join and meet have only been applicable to unifiable pairs of trees, in the sense of branch configuration.
If we could amend the definitions of these, preserving thetwo requirements mentioned in Section 2.4, the applicability would be greatly improved. If we could provide the join
and meet operations satisfying the absortption law and theparallelism of distance in the previous section, the applicability of the operations would greatly increase, and we
could design more varieties of musical application. Thus,
we propose a new time-span tree representation and improved join and meet operations for it.
3.1 Ternary Branching Representation
In Section 3, we have proposed the framework in which atime-span tree is distinguished from a written score. Now,
disregarding join of two melodies on a score, we introduce a ternary-branching tree, which represents the superimposition of the left-branching and right-branching binary trees. A new representation for a time-span tree isintroduced, shown in BNF as follows:nt::= p | c(n , t , t )::= ⊥ | p | c(n , t , t )

Proceedings ICMC|SMC|2014

F ¢Q² ¢W²⊥F ¢Q²⊥ ¢W²

F ¢Q² ¢W²¢W²

¢W²

¢Q²

¢W² ¢Q² ¢W²

¢Q²

¢W²

14-20 September 2014, Athens, Greece

Figure 4. Three Node Forms in Novel Representation ofTime-Span Tree
Symbol p means a pitch event as a terminal symbol, and ⊥the bottom which means the identify element for the join
operation. Pitch event p contains the information of pitch,maximal time span, and correspoding note on a score. nand t stand for a time-span tree; t can be ⊥, while n
is not. ⊥ may occur only at the second or third place, not at
the first. Term c(n , t , t ) represents a node of a timespan tree; the first place of the term n represents a primary branch, the second place (first t ) a secondary left
branch, and the third place (second t ) a secondary rightbranch (Fig. 4).
The idea here is that node c(n , t , t ) may be synthesized by the joining of unmatched-branching trees andjoining with fully-instantiated tree c(n , t , t ). The newtree representation enables the join operation to yield aproper result for those cases which have thus far not been
unifiable. The joining of unmatched-branching trees comprises cases such as join(c(n , t , ⊥), c(n , ⊥, t )) (theupper part of Fig. 5) and join(c(n , t , t ), c(n , ⊥, ⊥));
joining with fully-instantiated tree c(n , t , t ) comprises cases such as join(c(n , t , t ), c(n , t , t ))and join(c(n , t , t ), c(n , ⊥, t )). Simply, the join

operation recursively computes the argument-wise join. The
ternary branching representation can be regarded as the superposition, abstracting the distinction of left-/right- branching, of a binary tree, not as a node having three branches.
Moreover, the lower part of Fig. 5 shows the calculationof meet in one of the formerly-nonviable cases. Similarly,
the meet operation recursively computes the argument-wisemeet. Thus, in this case, the meet operation takes into account only the primary branches, ignoring secondary branches, which is equivalent to the treatment in the previous research [4].
Note that the ternary-branching tree representation introduced here is distinguished from a ternary branching timespan tree which may occur in ternary meter 2 . The ternarybranching appears only when we calculate join operation
tentatively. There is still the necessary condition that weare able to calculate the join operation, which is a joined
maximal time-span being concatenated, otherwise the result is undefined. Let [b, e] be a time-span beginning at b
and ending at e; we may assume the join of [1, 3] and [2, 4]
would be the connected interval of [1, 4] while that of [1, 2]
and [3, 4] would remain as two separated intervals. Incidentally, the meet of [1, 3] and [2, 4] is [2, 3], and that of[1, 2] and [3, 4] is undefined, not as ⊥.
3.2 Theoretical Properties
To introduce the proper join, we assume some useful concepts of the time-span tree beforehand.
Definition 1 (Structural Equivalence) Given a node c ina time-span tree representation,c(p, ⊥, ⊥) ≡ p
where p is atomic, either a pitch event or ⊥MRLQ^ƵďƚƌĞĞ
^ƵďƚƌĞĞ


PHHW^ƵďƚƌĞĞ
⊥^ƵďƚƌĞĞ

MRLQ^ƵďƚƌĞĞ
^ƵďƚƌĞĞ


PHHW

^ƵďƚƌĞĞ
^ƵďƚƌĞĞ
^ƵďƚƌĞĞ
^ƵďƚƌĞĞ
^ƵďƚƌĞĞ
It follows that ⊥ is equivalent to c(⊥, ⊥, ⊥),c(c(⊥, ⊥, ⊥), ⊥, ⊥), c(⊥, c(⊥, ⊥, ⊥), ⊥), and so on. As aresult, there are an infinite number of such trees equivalentto ⊥. For example in the lower part of Fig. 5, let t be a tree,
then c(t, ⊥, ⊥) cannot be rewritten to t if t is not atomic.
Suppose pi means a pitch event, then c(c(p1 , ⊥, ⊥), ⊥, p2 )can be rewritten to c(p1 , ⊥, p2 ).
As we have extended the new representation of time-spantree with ternery branching node c and the structural equivalence rule, we can similarly extend all the definitions on
reduction path, reduction distance, total maximal time-span,
and the lemmas on uniqueness of reduction distance that
we have developed in Sections 2.1, 2.2 and 2.3. Finally,
we can prove the theorem on triangle inequality of distancewith the new representation of time-span tree, although wewould like to omit the details of the definitions and theproofs of the lemma and the theorem.
We show an example in which given two pieces, the join
and meet are calculated (Fig. 6). The two pieces are takenfrom the Mozart’s variations K.265/300e ‘Ah, vous diraije, maman’, the variations No.2 and No.5. Actually, inthe process of calculating the join and meet operations of^ƵďƚƌĞĞ
^ƵďƚƌĞĞ
^ƵďƚƌĞĞ


⊥

2 Since GTTM restricts a time-span tree to a binary tree, a ternary
branching time-span tree is not allowed [5, pp.326-330].

Figure 5. Join and Meet of Unmatched-Branching Trees- 994 -

Proceedings ICMC|SMC|2014

14-20 September 2014, Athens, Greecejoin(No.2, No.5) (822)œ œ œ œœ œ œb n œ œœœ œœb n œ œœb n œ œœ # œœ œ
& 24 œœœœœœ œœœ œœœœ œœœœ œœœœœ œœœ œœœœœœ œœœœ œœœœ œœœœœœ œœ œ œ
r

A
A
d A
b n œ œœœ œb œn œ œœœ œ# œœ
b n œ œœr
& 24 œœœœœœœœ œœœœœœœœ œœœœœœœœ œœœœœœœœ œœ
œ œ
œ
œ œ
œ œ
A
A
œ
A
A
UAr
A
d 
A
A
meet(No.2, No.5) (576) Ar

No.2 (744)& 24 œ œ œœ œœ . œJ
œ

œ.

œ œ œ
J

œ œ

No.5 (654)œ œ œ œ œ œ œ œ œ œ œ œ œ
2& 4 œœœœ œœœœœœ œ œ œ œ
œ
œ œ
œ

œ œ
œ

œ

Figure 6. Parallelogram Composed of Variations No.2 and No5, join and meet. The values in the parentheses are total
maximal time-spans.
these two time-span trees, join and meet of unmatchedbranching ones occur nine times, respectively, and the distances via join and meet, d and d , are the same. The
value in the parenthesis shows the total maximal time-spanof each time-span tree; according to the definition of distance, we obtain d = (822 − 744) + (822 − 654) = 246
and d = (744 − 576) + (654 − 576) = 246. Notice thatthe four time-span trees form a parallelogram because thelengths of the opposite sides are equal respectively. Then,
we have confirmed the lemma on uniquness of reduction
distance in the proposed framework.

join(α, β)
E

͗

D

σ

σ
β
E

͗

α

E

͗

D

D

meet(σA, σB)
Figure 7. Morphing Algorithm

4. MORPHING ALGORITHM
Morphing is an algorithm to find an intermediate graphicalimage, given two images. We provide a similar methodology to compose an intermediate piece of music, given two
music pieces; especially given two existing variations witha common theme as in the paper [4]. Let σA and σB be
two pieces of music, and σC be an expected result of morphing; we require that σC should reside at an internallydividing point of σA and σB by N : M . The ratio M : N
means the one in terms of the total sum of maximal timespans (denoted as tmts in Section 2.2). Notice that there
are infinitely many σC ’s such that the ratio of the distance
between σA and σC to that between σC and σB is M : N
because σC resides on so-called Apollonian circles. Thus,
we should restrict σC to the one that resides at the shortest
distances from σA and σB , respectively.
Our morphing algorithm is shown in Fig. 7, consisting of:
1. Find such a reduction α of σA that divides σA andmeet(σA , σB ) by the ratio of N : M in terms of thegiven distance.
2. Find such a reduction β of σB that divides σB andmeet(σA , σB ) with the ratio of M : N .
3. join α and β.
We see that four time-span trees α, β, meet(σA , σB ), andjoin(α, β) also form a parallelogram as in Fig. 6. Apparently, in terms of the distance between σA and σB , wehave d(σA , σB ) = d(σA , join(α, β)) + d(join(α, β), σB ).

- 995 -

Moreover, tmts(σA ) ≤ tmts(join(σA , σB )) ≤ tmts(σB )holds if tmts(σA ) ≤ tmts(σB ).
We mention three points in implementing the morphingalgorithm. The first is related to the fact that our current
framework disregards matching of pitch events; the reduction operation takes only the information of the configuration of time-spans. Although we omit the technical details,
for obtaining the appropriate values of α and β, we prefer
to avoid the ratio N : M near to 1 : 0 or 0 : 1.
The second is related to rendering of the fully-instantiatednode c(n , t , t ), which can be regarded as the superimposition of the differently-branching nodes of two binary trees, not as a node having three branches. In thecurrent implementation, a fully-instantiated node is simply rendered as a chord of two notes, that is, sounding
both at the same time. Otherwise, for instance, it could
be rendered as a transformation of the superimposed timespans 3 .
The third is rendering itself. In the present rendering algorthm, a maximal time-span is basically considered as aline segment in a piano roll score, and the time-spans at alower level (closer to leaves) overwrites those at a higher
level. Thus, it may occur that the entirety of the maximal
time-span is overwritten by the lower-level pitch events;that is, even though a pitch event is quite salient, that pitch
event may become inaudible, or its duration assigned on areal score may be very short.
3 It is like a transformation head [5, p.155].

Proceedings ICMC|SMC|2014

No.1

œœ# œœœœœœ b œn œœ
& 24 œœœœœœœ

14-20 September 2014, Athens, Greeceœ œ œ œ œ œ œœœ œ œ œ œœ œœœ œ œ œœœ œ œ œ œœœ œ œœœ œ œ ˙

Ϭ͘ϴ

EŽ͘Ϯ

Ϭ͘ϲ

No.1&No.2

œ
œœ
œœ
œ
œœœœœ œ œœ# œœœœœœœœœœœœ b œ n œœ œ œ œ œ œœœ œœ œ œ œ œ œ œ œ œ œ œ œ œ
2
œ
œ
œ
&4 œ
œ
œ œ œœ œ
œ
œ
œœ
œ
œ
œ

Ϭ͘ϰ

EŽ͘ϱ

Ϭ͘Ϯ

No.2

㻙㻜㻚㻤

œ œ œ œ œ œœ2œ
& 4 œœ œœ œ œ œ œ œ œ œ œœ œ œ œ œ œ œ œ œ œ œ œ œœœ œ œ œœœ œœ œ
œ œ& 24 œ œ
œ

œ œ

œœ

No.5&No.1

œ œ

œœ

œ œ

œœ

œ

œ
œ œ

œ œ

œ

㻙㻜㻚㻠

㻙㻜㻚㻞

㻜

㻜㻚㻞

ͲϬ͘ϰ

㻜㻚㻠

㻜㻚㻢

㻜㻚㻤

EŽ͘ϭ

EŽ͘ϱΘEŽ͘ϭͲϬ͘ϲœ œ œ œ œ œ œœ œ œ œ
œ
œ œ
œ

œ
œ
œ
œ œ œ œœ œ œ œ œ œ œ œ œ œ œ œ œ œ . œ œ
2
&4
œ
œ
œ
œ

㻙㻜㻚㻢

Ϭ

ͲϬ͘Ϯ

No.2&No.5

No.5

EŽ͘ϭΘEŽ͘ϮEŽ͘ϮΘEŽ͘ϱb n œ œœœ œb œn œ œ œ œ# œœ
b n œ œœ& 24 œœœœœœœ œœœœœœœœ œœœœœœœœ œœœœœœœ œœ
œ
œ
œ
œ œ
œ
œ œ
œ œ
œ

Figure 9. Relative Distance Among Variations and Morphed Melodies According to the Impression of Human ListenersœFigure 8. Variations No.1, No2, and No.5, and morphed
melodies between them
5. EXPERIMENT AND RESULTS
The morphing algorithm is implemented in SWI-Prolog[10]. The set piece is Mozart’s variations K.265/300e ‘Ah,vous dirai-je, maman’. The piece consists of the famous
theme and twelve variations of it. In our experiment, we
take variations No.1, 2, and 5 as the sources for morphing,and excerpt the first eight bars (Fig. 8). We have chosen
these three variations because for every pair of these two
we can calculate the result of join, that is, joined maximal time-spans are all concatenated. To make comparison
easy, the morphed melodies generated by the improved algorithm are shown between the variations. For example, inthe figure, “No.2&No5” means the morphed melody at the
midpoint of variations No.2 and 5.
For the similarity assessment of the morphed melodies byhuman listeners, six university students (2 females and 4
males) participated in our study, four of whom have experience of playing music instruments for five years or more.
We use the method similar to the previous research[4]. An
examinee listens to all pairs m1 , m2 in random orderwithout duplication, where m{1,2} is either variations No.1,
No.2, No.5 and the morphed melodies between them, suchas No.1&No.2. Every time he/she listens to it, he/she is
asked “how similar is m1 to m2 ?”, and rates it using oneof following five grades: quite similar = 2, similar = 1,
neutral = 0, not similar = −1, and quite different = −2.
At the very beginning, for cancelling the cold start bias, every examinee hears the theme and twelve variations (eightbars long) without rating them. In addition, when an examinee listens to and rates pair m1 , m2 , he/she should
try the same pair later to avoid the order effect. Finally, theaverage ratings of each examinee are calculated and thenthe average for all the examinees is determined.
The experimental results are obtained in the distance-ma-- 996 -

trix between variations No.1, No.2, No.5 and the morphed
melodies between them at first. Since it is difficult to examine the results as they are, we employ multidimensional
scaling (MDS) [12] to visualize the results (Fig. 9). To explain briefly, MDS plots items on a coordinate plane so thatthe more similar items are, the closer they are.
In terms of Nos. 1 and 2 pair and Nos. 1 and 5 pair,the morphed melodies are plotted at the midpoint of theirsource variations almost as expected. In contrast, the position of No.2&No.5 is problematic. As can be seen in
Fig. 8, No.2&No.5 is the internally dividing point of No.2
and No.5 by 1: 1, and the number of notes of No.2&No.5
is approximately the average of No.2 and No.5. However,No.2&No.5 is almost entirely made of eighth notes, and as
the result of join, many of the notes which have the same
pitch or which sound at the same time. Consequently, it
can be thought that the human impression of No.2&No.5
is closer to that of No.5.
6. CONCLUSIONSIn this paper, we have proposed the time-span tree representation and the join operation, applied to two time-spantrees. In general, the result of the join operation on two
arbitrary input pieces of music is not obvious. That is, it
is not straightforward to construct the valid join satisfying
the basic properties such as the absorption law that is consistent with the notion of reduction provided by GTTM.
We explained that we strictly distinguished the tree representation from the actual music represented on scores.
By use of the join and meet operations, we implemented
an automatic morphing system in Prolog, and composed
virtual variations of K.265/300e by Wolfgang Amadeus
Mozart from existing variations. Since the distance between time-span trees defined in the paper satisfies the properties desired for morphing, we can identify the internal
dividing point of time-span trees σA and σB by N : M
as if we draw a figure using a triangle ruler and a compass (Fig. 7). We have evaluated these synthesized variations according to the impression of human listeners, andProceedings ICMC|SMC|2014

14-20 September 2014, Athens, Greece

found an interesting correspondence between the theoretical distance and psychological distance. As a result, wehave shown that the use of join and meet operations in
our algebraic framework could suffice to produce viable
Mozartoid variations.
We think the tree distance proposed should be only applied to short pieces, for instance, consisting of eight tosixteen bars; otherwise, we need to consider whether or
not a single tree exists for a longer piece of music. In
effect, our definition of distance strongly depends on thestrength of heads, and if these heads are changed it affects
the distance inadequately. Investigating the relationships
between the adequacy of the distance versus the length ofmusic piece should be our immediate future work.
We can imagine many possible algorithms for rerendering besides the current one as we discussed in Section 4.
For example, a rendering algorithm may take into accountthe original notes from which the relevant time-spans arederived. Another one may employ the technique of casebased reasoning with a database consisting of the melody/ time-span tree pairs. On the other hand, rendering canbe viewed as the inverse process of the GTTM analysis asshown in Fig. 3. Here let us consider the piece obtainedby the following two steps: the GTTM analysis builds atime-span tree from an original piece, and a rendering algorithm synthesizes the resulting piece from a time-span
tree. Then, a pair of the GTTM analysis and a rendering
algorithm that restores the original piece may be proper.
Therefore, we think that a rendering algorithm should always be investigated with GTTM analysis.

[7] G. Nierhaus, Algorithmic Composition, SpringerVerlag (2009).
[8] D. Rizo-Valero, Symbolic Music Comparison with Tree
Data Structure, Ph.D. Thesis, Universitat d’ Alacant,
Departamento de Lenguajes y Sistemas Informatı́cos,
2010.
[9] C. Roads, The Computer Music Tutorial, The MIT
Press (1996).
[10] SWI-Prolog, http://www.swi-prolog.org/ Accessed on April 1, 2014.
[11] S. Tojo and K. Hirata, Structural Similarity Based on
Time-span Tree, Proceedings of CMMR 2012, pp.645660, 2012.
[12] Wikipedia, Multidimensional scaling,

Acknowledgments
This work was supported by JSPS KAKENHI Grant Numbers 23500145 and 25330434.
7. REFERENCES[1] M. Hamanaka, K. Hirata, S. Tojo, Implementing “AGenerative Theory of Tonal Music”, Journal of New
Music Research, 35(4), 249–277, 2007.
[2] M. Hamanaka, K. Hirata, S. Tojo, Melody Morphing
Method Based on GTTM, Proceedings of ICMC 2008,
pp.155-158, 2008.
[3] K. Hirata, S. Tojo, and M. Hamanaka, Melodic Morphing Algorithm in Formalism, In Proceedings of
MCM2011, also in LNAI6726, Springer, pp.338-341,
2011.
[4] K. Hirata, S. Tojo, and M. Hamanaka, Cognitive Similarity grounded by tree distance from the analysis ofK.265/300e, Proceedings of CMMR 2013, pp.415-430,
2013.
[5] F. Lerdahl and R. Jackendoff, A Generative Theory ofTonal Music, The MIT Press, Cambridge, 1983.
[6] A. Marsden, Generative Structural Representation of
Tonal Music, Journal of New Music Research, 34(4),409–428, 2005.

- 997 -

http://en.wikipedia.org/wiki/
Multidimensional scaling, Accessed on April 1,

2014.