# Tuning of a Knowledge-Driven Harmonization Model for Tonal Music

Mariusz Rybnik, Wladyslaw Homenda

► To cite this version:

Mariusz Rybnik, Wladyslaw Homenda. Tuning of a Knowledge-Driven Harmonization Model for Tonal Music. 11th International Conference on Computer Information Systems and Industrial Management (CISIM), Sep 2012, Venice, Italy. pp.326-337, 10.1007/978-3-642-33260-9_28. hal-01551720

HAL Id: hal-01551720

https://inria.hal.science/hal-01551720

Submitted on 30 Jun 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Tuning of a Knowledge-driven Harmonization Model for Tonal Music

Mariusz Rybnik¹ and Wladyslaw Homenda²

¹ Faculty of Mathematics and Computer Science, University of Białystok, ul. Sosnowa 64, 15-887, Białystok, Poland
mariuszrybnik@wp.pl

² Faculty of Mathematics and Information Science, Warsaw University of Technology, Pl. Politechniki 1, 00-661, Warsaw, Poland
homenda@mini.pw.edu.pl

**Abstract.** The paper presents and discusses direct and indirect tuning of a knowledge-driven harmonization model for tonal music. Automatic harmonization is a data analysis problem: an algorithm processes a music notation document and generates specific meta-data (harmonic functions). The proposed model could be seen as an Expert System with manually selected weights, based largely on the music theory. It emphasizes universality - a possibility of obtaining varied but controllable harmonies. It is directly tunable by changing the internal parameters of harmonization mechanisms, as well as an importance weight corresponding to each mechanism. The authors propose also indirect model tuning, using supervised learning with a preselected set of examples. Indirect tuning algorithms are evaluated experimentally and discussed. The proposed harmonization model is prone both to direct (expert-based) and indirect (data-driven) modifications, what allows for a mixed learning and relatively easy interpretation of internal knowledge.

**Keywords:** harmonization, data analysis, expert system, musical work, supervised learning, tonal music

## 1 Introduction

*Harmony* is an important element of tonal music, it defines a vertical relation between notes [1], and by definition is opposed to *melody* - a horizontal succession of notes in a specific voice. In fact, however, the harmonic relations of the leading melody (regarding mono- or homophony) are largely depending on the horizontal succession of notes and *melodic intervals* between them. Harmonic *passages* that follow harmonic *chords* (with smaller or larger deviations) can be frequently detected in a melody. Similar, but obviously much stronger harmonic relations are to be found in an *accompaniment*, where the melody is less (or even not at all) important, as the main goal is to define a background for the leading melody. The obvious exception to these assumptions are polyphonic musical works. They tend to cultivate two or more independent voices, that compete for attention,
but also have to cooperate harmonically, usually indicating strong harmonic
relations.

*Automatic harmonization* can be seen as a problem crossing two different areas: 1) theoretical music knowledge and 2) formal mathematical computations - presented in form of algorithms or slightly less formal Artificial Intelligence solutions. From the technical point of view it could be seen as analyzing of a music notation document and producing meta-data (harmonic functions). Over the years, many various approaches and techniques were used to solve the problem (or similar). The most popular paradigms are: Expert Systems [2][3], Neural Networks [4], Constraints and probabilistic approaches [5] [6] [7], evolutionary algorithms [8]. Due to excessive complication and uncertainty, related to such sensitive subject as music (feelings and sensitivity still tend to evade scientific approaches), no approach seems to fully explore and describe the subject. In fact, some of the approaches focus on a particular style of musical works, as for example typically Baroque pieces of J. S. Bach in [4]. This allows for relatively narrow specialization and therefore eases the algorithmic description of the problem.

Frequent limitations for above-mentioned approaches seem to be: lack of uni-
versality (results limited to a specific area), an unpredictability, a need for a large
learning examples database and an extensive amount of calculations. The au-
thors' approach is mostly based on an advanced theoretical music knowledge,
especially in the area of harmonization, and is aimed at solving these disadvan-
tages. The authors propose a model that may be customized at various levels
and does not need any learning examples, as they are replaced by an expert
knowledge of harmony, incorporated in model. It may be defined initially as Ex-
pert System with manually selected weights, however the authors also propose
a scheme for tuning the general harmonization model. With enough knowledge
of *harmony* one can also directly modify the model using internal parameters.

In this paper the authors concentrate on tuning procedures of their harmo-
nization model introduced in [9] and extended in [10]. The paper is organized into
five sections; Section 1 being an introduction. Section 2 presents basic concepts
concerning harmony and harmonization in tonal music. Section 3 describes the
harmonization model and explains mechanisms used. Section 4 discuses various
tuning possibilities of a proposed model and presents a sample of experimental
results. Finally, Section 5 concludes the paper, discuses configuration, universal-
ity properties and suggests future work.

## 2 On harmony in tonal music

The process of creating accompaniment to a *homophony* (lone melody - a single-
layered progression of sounds in time) can, in practice, be divided into two
phases. The first is *harmonization* (determination of harmonic functions and
chords corresponding to them), the second phase is creation of accompaniment
using previously determined chords. It is important to stress, that in vast num-
ber of cases there is no single, ultimate harmonization for a given melody. There
are many possibilities, from the most simple and obvious to the highly compli-
cated and non-trivial ones (e.g. improvised music, jazz). The decision depends
mostly on a style of music. It also relies on capabilities of the available instru-
ment/orchestration, and the abilities of performers/harmonizers. There are also
cases when no additional information is needed to perform the music, except for
the leading melody and harmonic functions. A common example is an improvised
à vista accompaniment to songs played on a guitar or a piano.

## 2.1 The tonal harmony

For the purposes of this paper the authors have focused on a *tonal system* with two basic *scales*: *major* and *minor* (*natural minor* as well as *harmonic* and *melodic* modifications). The authors’ considerations and experiments are based on seven diatonic *harmonic functions* (built on 1<sup>st</sup>, 2<sup>nd</sup>, 3<sup>rd</sup>, 4<sup>th</sup>, 5<sup>th</sup>, 6<sup>th</sup> and 7<sup>th</sup> grade of major/minor scales) with common modifications: adding the *seventh*, *ninth*, and *sixth* (all three minor and major).

The harmonic relations in tonal system tend to be well defined in terms
of *consonants* (chords 'pleasant' to human ear, stable) and *dissonances* (chord
slightly 'unpleasant', unstable, introducing a tension that needs to be *resolved*
to consonants). The authors have omitted alterations and higher intervals (alike
*eleventh*, *thirteenth*), that may be considered as *dissonances* meant to *resolve*,
rather than intrinsic harmony.

## 3 Proposed harmonization model

The proposed harmonization model is based on several mechanisms that closely
follow music theory:

1. **Particular note can have various harmonic importance** (based on the note's relative length, the notes placement in measure, surrounding notes, volume, etc.);

2. **Each note excites (fits to) several harmonic functions**, based on pitch and function components taken into consideration (components higher than 9th are usually very rare and considering them is very difficult);

3. **Some harmonic functions are more likely to occur than others** (the simplest example being *Tonic* - a base and consolation for vast majority of tonal music, therefore usually occurring most frequently). It is preferable to prioritize the commonly used functions;

4. **Some specific successions of harmonic functions are more or less probable**, therefore it is possible to prioritize the more likeable (frequent) successions (e.g. *Dominant* → *Tonic* or *Tonic* → *Subdominant*).

The mechanisms are implemented in an independent way (when possible) and
weighted using a standard range [0; 1], in order to easily configure (or eliminate)
their degree of influence.
**3.1 Data representation of harmonic functions**

Each harmonic function is stored in a structure that contains a vector of *Harmonic Function Strengths* (since we consider 7 different diatonic functions its length is 7) and a corresponding vector of function modifiers. *Harmonic Function Strength* is a factor used to determine probability of occurrence of a specific diatonic harmonic function. In the authors' experiments the function modifiers are mostly limited to *sevenths*, with occasional *ninths* and in rare cases to *sixths*. *Sixth* is present in so-called *Chopin's chord: Dominant* with *seventh* and natural or augmented *sixth* instead of *fifth*, resolving down to the first degree of scale. The diatonic function with modifiers in a given *key* is equivalent to chords e.g. *Tonic* with modifier *seventh* in key G-major is equivalent to G<sup>7</sup>. Some modifiers cover others, e.g. a chord with a *ninth* is a variation of a chord with a *seventh*. Modifiers can be seen as subclasses of the main class (basic diatonic function).

For simplification reasons the authors have decided to determine the har-
monic functions in constant intervals: twice per measure in the case of 3 or 4
beats and once per measure in the case of 2 beats. This is sufficient in most
cases, as in practice the harmonic functions rarely change more frequently. The
authors have also experimented with harmonic fragment being equal to the whole
measure. Musical piece is decomposed into *harmonic fragments* (defined as in-
divisible musical unit with a single diatonic function attached to each one of
them). The *Harmonic Function Strength* for a *harmonic fragment* is a sum of
*function excitations* (determined by note pitch) for all notes in the fragment,
according to varied degrees of *Note Importance*.

**3.2 The flow of the proposed model**

The flow of the proposed model regarding a single *harmonic fragment* is pre-
sented in Fig. 1, and proceeds as follows:

1. The harmony of a particular fragment is evaluated by examining all notes that it is composed of. Based on a set of rules from Section 3.3, every note is attached with a *Note Importance* value.

2. Each note excite (add a certain value to *Function Strengths* vector) several corresponding diatonic functions by being a specific chord component, with degree defined by the *Function Excitation Matrix* (Section 3.4).

3. After all the notes are processed, the vector of *Function Strengths* is elementwise multiplied by a *Function Popularity Vector* (Section 3.5). This primarily serves as a method of favouring the more popular functions. The *Function Popularity Vector* can also be a way to get a rich and uncommon harmony.

4. A *Function Successions* mechanism is applied after the individual processing of all of the harmonic fragments is complete. *Function Succession* matrix (Section 3.6) defines the degrees of desire for a specific succession of harmonic functions. They serve as additional modifiers (regarding direct predecessor and successor) for vectors of the *Function Strengths*. Every fragment is modified twice, first as a predecessor and the second time as a successor; only
<figure><img src="image_2.png" /><figcaption>Fig. 1. Harmonization model flow</figcaption></figure>

the outmost fragments are changed once. The modifications are calculated for every possible function succession (a total of 49 combinations is possible for 7 recognized functions, the modifiers are not taken into account). They are multiplied by both corresponding function strengths and additionally by a 1/12 factor what potentially keeps them in the original *Function Strengths* range.

5. A classification decision is made for each *Harmonic Fragment*. It is based on values of *Function Strengths*. *Winner-takes-all* decision was used for simplicity reasons; the classification-like relations are already hidden inside the above mentioned mechanisms. It is important to stress that any classifier can be used for this purpose (including direct use of function subclasses, described in Section 3.1. Determination of classifier for this purpose is a large topic exceeding the scope of the paper. The winning *diatonic function* is translated into *chord*, regarding the *key*. Finally it is attached with chord modifiers (e.g. *seventh*, *ninth*), that were stored independently during *Excitation*.

Each omittable mechanism (*Note Importance*, *Function Popularity*, *Function Succession*) is attached with an overall *Global Configuration Weight* from the range [0; 1] (Fig. 1). It can be used to fine tune or disable the influence of a specific mechanism on the final harmonization result. *Excitation* is an essential mechanism to obtaining *Function Strengths*, therefore it may not be weighted in a similar manner. The mechanisms are based on matrices (*Function Excitation*,
*Function Succession*), vectors (*Function Popularity*) or rules (*Note Importance*) that offer substantial, direct and indirect configuration possibilities. The following subsections describe in detail the above-mentioned mechanisms.

## 3.3 Note importance determination

*Note Importance* may be determined by:

- **the length of note** - longer notes have, in general, greater influence on the harmony. The note length is used as an initial value for further importance weighting. Values corresponding to neighboring note lengths (alike *eighth* note and *quarter* note) have common ratio, empirically stated as 5/6.

- **position in measure** - notes at inherently accented parts of measure - *on-beat* - are very important. Notes at generally unaccentuated parts of measure - *off-beat*, are less significant. Notes occurring in-between these main beats are harmonically even less important.

- **notes that are at the end of ties have minor contribution to the harmony;** when played on string instruments (e.g. grand piano, upright piano, guitar) they are not hit again. They are therefore relatively quiet.

- **notes that are easily-heard by human ear are placed in extreme voices** (highest and lowest notes); notes in the middle voices are slightly harder to hear and therefore contribute less to the local harmony;

- **accentuation increases the volume** and therefore harmonic value of the note.

Homophony harmonization was chosen for the experimental evaluation, therefore the authors have determined a set of *Note Importance* rules described in Table 1. In this case, rules regarding voice position are irrelevant, and are not taken into consideration. These rules would be however important for music works with many *voices*. Corresponding *note length* from upper part of the Table 1 is taken as an initial value of *Note Importance*. Initial values for non-standard note lengths (e.g. a half note with a dot) are determined proportionally. The weight corresponding to the first fitting rule from lower part of the Table 1 serves as the multiplier for the initial value.

## 3.4 Implementation of Functions Excitation by notes

A well-defined *diatonic function* contains at minimum three components (typically *root note*, *third* and *fifth*). More complicated diatonic functions contain also additional or altered intervals. Determination of exact *pitches* of these function components requires detection of the piece *key* and reading of the current *key signature*.

Two matrices of *Function Excitation* are used to define the excitations of diatonic functions with notes (from the melody or the accompaniment). The matrix for major scales is presented in Table 2. Each diatonic function is excited by the particular matching note to a specified degree. This, obviously, is relative to the
<table><caption>**Table 1.** Determining Note Importance</caption><thead><tr><th>Note length</th><th>Initial value</th></tr></thead><tbody><tr><td>Whole note</td><td>1.440</td></tr><tr><td>Half note</td><td>1.200</td></tr><tr><td>Quarter note</td><td>1.000</td></tr><tr><td>Eighth note</td><td>0.833</td></tr><tr><td>Sixteenth note</td><td>0.694</td></tr><tr><td>Thirteenth-second note</td><td>0.579</td></tr></tbody><thead><tr><th>Condition</th><th>Importance weight</th></tr></thead><tbody><tr><td>Note at the end of a tie</td><td>0.4</td></tr><tr><td>Note on the 1<sup>st</sup> beat</td><td>1.0</td></tr><tr><td>Note on the 3<sup>rd</sup> beat</td><td>0.9</td></tr><tr><td>Accentuated note</td><td>0.8</td></tr><tr><td>Note on the 2<sup>nd</sup> or 4<sup>th</sup> beat</td><td>0.7</td></tr><tr><td>Other notes</td><td>0.6</td></tr></tbody></table>

pitch of the note in question, and to the pitches of the notes occurring in the tonal functions themselves. The authors have defined the weights of excitation in the range of [0, 1], assigning:

- [0.8; 1.0] - greater degrees to the common chord components (e.g. *root note*, *third* and *fifth*);

- [0.4; 0.6] - moderate degrees to the less common components: *sixth*, *seventh*, *ninth* with occasional modification of natural *third*. Making major chords from naturally minor chords using augmented *third* serves frequently as *tonicization* - a local *Dominant* → *Tonic* resolution);

- [0.1; 0.2] - small degrees to the rare modifications of natural *sixth*, *seventh* and *ninth*.

<table><caption>**Table 2.** Excitation weights for major scales</caption><thead><tr><th rowspan="2"></th><th colspan="11">Absolute shift from function <i>root note</i>[in semitones]</th></tr><tr><th>0</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>11</th></tr></thead><tbody><tr><th><i>D</i><sub>7</sub></th><td>1</td><td>0</td><td>0</td><td>0.9</td><td>0</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th><i>T</i><sub>6</sub></th><td>1</td><td>0</td><td>0</td><td>0.9</td><td>0.1</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0.4</td><td>0</td></tr><tr><th><i>D</i></th><td>1</td><td>0.5</td><td>0.2</td><td>0.1</td><td>0.9</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0.4</td><td>0.6</td><td>0.1</td></tr><tr><th><i>S</i></th><td>1</td><td>0</td><td>0</td><td>0.1</td><td>0.9</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0.4</td><td>0</td></tr><tr><th><i>T</i><sub>3</sub></th><td>1</td><td>0</td><td>0</td><td>0.9</td><td>0.1</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0.4</td><td>0</td></tr><tr><th><i>S</i><sub>2</sub></th><td>1</td><td>0</td><td>0</td><td>0.9</td><td>0.1</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0.4</td><td>0</td></tr><tr><th><i>T</i></th><td>1</td><td>0</td><td>0</td><td>0.1</td><td>0.9</td><td>0.1</td><td>0</td><td>0.8</td><td>0</td><td>0</td><td>0.4</td><td>0</td></tr></tbody></table>

A similar matrix has been prepared for minor scales. Due to space limitation it is not presented in this paper. The obvious differences in reference to major
scales are the natural qualities of diatonic functions. The less straightforward dissimilarity is a frequent conversion from naturally minor *Subdominant* and *Dominant* to major. This commonly occurs in a *melodic* variation of scale, and sometimes, in *harmonic* variations.

## 3.5 Function popularity

The goal of determining and applying weights corresponding to *Function Popularity* is to directly prioritize frequently occurring tonal functions (e.g. *Tonic*, *Subdominant*, *Dominant*). This is a direct way to make them more frequent than less popular functions, as inherently specified by music theory. The goal can be indirectly obtainable by using lower coefficients in the *Functions Succession* matrix (described in Section 3.6). Obviously, the direct method makes controlling the process much easier. For experimental studies the authors have used the *Function Popularity* vector, specified in Table 3.

<table><caption>Table 3. Function Popularity vector weights</caption><thead><tr><th colspan="2">Tonal function</th></tr><tr><th>Name</th><th>Weight</th></tr></thead><tbody><tr><td><i>T</i></td><td>1.0</td></tr><tr><td><i>S</i><sub>2</sub></td><td>0.6</td></tr><tr><td><i>T</i><sub>3</sub></td><td>0.7</td></tr><tr><td><i>S</i></td><td>0.9</td></tr><tr><td><i>D</i></td><td>0.95</td></tr><tr><td><i>T</i><sub>6</sub></td><td>0.65</td></tr><tr><td><i>D</i><sub>7</sub></td><td>0.2</td></tr></tbody></table>

## 3.6 Succession of harmonic functions

The succession of harmonic functions (horizontal relations between neighboring harmonic functions) is implemented using encouragement of more probable combinations, with moderation of less likable ones. This is done using a *Function Succession* matrix. The matrices proposed for major and minor scales are presented in Table 4. The matrix for minor scale is similar and due to space limitations not presented here.

The matrices values were determined in order to prioritize the most likable successions. The exemplary common successions are: *Tonic* into *Subdominant*, *Subdominant* into *Dominant* and *Dominant* into *Tonic* - cadence (the simplest and most common). Another example is less common *deceptive cadence* (*Dominant* into *Tonic*<sub>6</sub>). The matrices also support *tonicization* (e.g. succession from *Subdominant*<sub>2</sub> into *Dominant*). It is worth mentioning that the most supported
<table><caption>**Table 4.** *Functions Succession* weights for major scales</caption><thead><tr><th rowspan="2"></th><th colspan="6">Transfer into:</th></tr><tr><th>T</th><th>S<sub>2</sub></th><th>T<sub>3</sub></th><th>S</th><th>D</th><th>T<sub>6</sub></th><th>D<sub>7</sub></th></tr></thead><tbody><tr><th>D<sub>7</sub></th><td>0.5</td><td>0.3</td><td>0.5</td><td>0.2</td><td>0.9</td><td>0.1</td><td>1.0</td></tr><tr><th>T<sub>6</sub></th><td>0.4</td><td>0.8</td><td>0.3</td><td>0.8</td><td>0.7</td><td>1.0</td><td>0.2</td></tr><tr><th>D</th><td>0.9</td><td>0.4</td><td>0.7</td><td>0.3</td><td>1.0</td><td>0.8</td><td>0.3</td></tr><tr><th>S</th><td>0.7</td><td>0.4</td><td>0.4</td><td>1.0</td><td>0.9</td><td>0.5</td><td>0.1</td></tr><tr><th>T<sub>3</sub></th><td>0.3</td><td>0.2</td><td>1.0</td><td>0.5</td><td>0.6</td><td>0.8</td><td>0.1</td></tr><tr><th>S<sub>2</sub></th><td>0.2</td><td>1.0</td><td>0.2</td><td>0.5</td><td>0.9</td><td>0.4</td><td>0.2</td></tr><tr><th>T</th><td>1.0</td><td>0.4</td><td>0.3</td><td>0.8</td><td>0.9</td><td>0.6</td><td>0.1</td></tr></tbody></table>

quasi-succession is maintaining the current function (no change) with the maximum degree of support: 1.0. It allows more efficient handling of common cases, where harmonic functions change less frequently than the arbitrary *harmonic fragment* length (e.g. two beats). It is also important to mention that the support of succession occurs only between harmonically determinable *harmonic fragments*, excluding these that do not contain notes at all (only pauses), or contain only ongoing tied notes from previous beats. It is assumed that such fragments continue the previous harmonic function (which is a slight oversimplification as it does not have to be always true). The succession support occurs forwards and backwards for each possible succession, with a degree defined by the sum of products of the neighboring *Function Strengths* and a succession weight from the *Functions Succession* matrix.

# 4 Model tuning and experimental results

The authors have implemented the proposed model and tested it using music documents in MusicXML file format [11]. This section discusses various tuning possibilities and presents exemplary homophony harmonizations with the use of various tuning approaches.

The model has been applied to musical works containing a single melody (*monophony*) with no *key* changes (no modulations). It may be considered as a greater challenge (more harmonic uncertainty) than using musical works with accompaniment or several independent voices, where harmonic functions are generally easier to detect. The proposed harmonization model is viable for almost every musical piece, regardless of the number of voices, as long as it is maintained in a tonal system, e.g. uses either major or minor scale (with possible modifications like harmonic, melodic, Dorian, etc.). In case of many voices or chords, ideally, a customized *Notes Importance* determination is required (as described in Section 3.3), in order to detect and prioritize more important voices, and attenuate the less important ones. Efficient harmonization of a musical works with *key* changes (*modulations*) requires detection or indication of such changes, and an adequate update of tonal root and/or minor/major scale properties.

The authors propose the following model modifications:
1. **changing values of Global Configuration Weights** (respectively: *Note Importance, Function Popularity, Function Succession*), exemplary applications are: moderation of less popular functions, introduction of uncommon function successions;

2. **direct tuning** - changing values in the configuration matrices (*Function Excitation* matrix, *Function Popularity* vector, *Function Succession* matrix) and defining custom *Note Importance* determination rules. The initial values are meant to be universal, changing them directly influences the behavior of the model and produced results in a specific direction, but requires experience and advanced knowledge of harmony;

3. **indirect tuning by learning from examples** - modification of a part of proposed approach (*Function Popularity* vector and *Function Succession* matrix) using non-direct modification (tuning) of matrices. The *example* is a harmonized music piece: a music document with attached harmony functions corresponding to class labels. A simple statistical analysis of the occurring harmonic functions are used to update the values in the *Function Popularity* vector. Analysis of occurring harmonic function successions in the *example* are used to update *Function Succession* matrix. Total extinction of rare functions and successions may occur (depending on the harmony relations included in the examples), therefore the authors propose to limit the minimal values to a fixed threshold 0.05. The learning process is independent regarding the sequence of examples - similarly to *batch learning*, as *sequential learning* would prioritize the recent examples over the previous ones.

The results in Fig. 2 present alternative harmonizations produced using different tuning procedures. The fragment of musical piece (A. De Vita and H. Sharper - *Softly, as I leave you*) was harmonized with *harmonic fragments* set to 2 beats (half of measure). The first and second harmonies were obtained for original model with various values of *Global Configuration Weights* (respectively: *Note Importance, Function Popularity, Function Succession*): {0.4, 0.4, 0.4} in the first case, {0, 0, 0.8} in the second case. In the second case less popular functions are encouraged to appear. Third harmony was obtained by using *direct tuning of the model* (by changing *Function Excitation* matrix to produce more seventh and ninth chords, and *Function Popularity* vector to encourage rare functions. Fourth harmony was obtained by using *indirect tuning by learning from examples*, with ten examples being modern popular music pieces of unspecified genre. The learning set has relatively simple harmonies, therefore the harmonization results (produced by the tuned model) are also simplified.

Regarding indirect tuning: it is possible to tune the original model into a specific direction by training with a preselected set of *examples* (i.e. jazz pieces with complicated harmony or musical pieces from a specific period). Such approach requires a number of carefully selected examples and is problematic as musical styles are often ambiguous.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Alternative harmonizations using different tuning procedures</figcaption></figure>

# 5 Conclusion and future work

In this paper the authors have presented music theory based harmonization model with emphasis on model tuning. The main goal of the approach is universality but also control over harmonization process and results. Various possibilities of the initial model modifications are discussed with given examples.

## 5.1 Remarks on universality

The proposed model offers numerous configuration possibilities and universality: as it is not data-driven but knowledge-driven, it provides multiple levels of model control and, to some extent, results of harmonization. As opposed to the data-driven approaches (taught with examples) the authors' methodology does not need to be fed with large or representative sample of data (requiring data gathering, selection and frequently producing uncontrollable results). It may be used in many variants relying only on the theoretical knowledge. With modifications of underlaying configuration data one can achieve many valuable tasks, such as:

- defining the *Note Importance* rules tuned to various musical pieces;

- defining the *Function Excitation* matrix to generate rare or simple functions/modifications, resulting in a complicated or simplistic harmony;

- defining the *Function Popularity* vector to moderate, forbid or encourage specific tonal functions, directly limiting or increasing probability of their occurrence;

- defining the *Function Succession* matrix to moderate, forbid or encourage specific harmonic function successions;

- changing the *Global Configuration Weights* of harmonization mechanisms in order to tune the model to the specific needs and expectations;

- iterate through configurations in order to quickly generate various (possibly interesting) harmonies for the same musical work.

The authors propose also data-driven tuning of the initial model, what could provide more viable harmonizations or (with preselected set of examples) produce specific harmony style.
## 5.2 Future works

Future works in the area will be conducted in the following directions:

- continuous development and evaluation of the presented harmonization model;

- determination of configuration matrices and parameters for different styles of musical works (like jazz, classical music, popular music, etc.) using indirect tuning with preselected set of examples (what requires relatively large and representative examples database);

- development of the *Function Succession* mechanism with use of chord modifiers;

- development of the *Function Excitation* mechanism using also relations between notes, rather than independent notes;

- developing criteria for automatic evaluation of the obtained harmony;

- automatic parametrization of the harmonization model based on the above-mentioned criteria;

- further development of model tuning procedures, using well established machine learning paradigms, e.g. Artificial Neural Networks.

## Acknowledgement

This work is supported by The National Center for Research and Development, Grant no. N R02 0019 06/2009.

## References

1. K. Sikorski, *Harmony* [in Polish], Polskie Wydawnictwo Muzyczne, 2003.

2. D. Cope, *An expert system for computer-assisted music composition*, In Computer Music Journal, volume 11 (4), pp. 30-46, 1987.

3. K. Ebcioglu, *An expert system for harmonizing four-part chorales*, In Machine Models of Music, pp. 385-401, MIT Press, 1993.

4. H. Hild and J. M. W. Feulner, *Harmonet: A neural net for harmonizing chorals in the style of J.S. Bach*, In Advances in Neural Information Processing 4, 1992.

5. F. Pachet and P. Roy, *Musical harmonization with constraints: A survey*, In Constraints Journal, volume 6(1), pp. 7-19, Kluwer Publisher, 2001.

6. F. Pachet and P. Roy, *Mixing constraints and objects: a case study in automatic harmonization*, In TOOLS Europe 95, pp. 119-126, Prentice-Hall, 1995.

7. J-F. Paiement, D. Eck and S. Bengio, *Probabilistic Melodic Harmonization*, Lecture Notes in Computer Science 4013, pp. 218-229, 2006.

8. R. D. Prisco and R. Zaccagnino, *An evolutionary music composer algorithm for bass harmonization*, In Applications of Evolutionary Computing, LNCS, 2009.

9. M. Rybnik and W. Homenda, *Knowledge-driven Harmonization Model for Tonal Music*, Proceedings of the 4th International Conference on Agents and Artificial Intelligence (ICAART 2012), pp. 445-450, 2012.

10. M. Rybnik and W. Homenda, *Extension of Knowledge-driven Harmonization Model for Tonal Music*, 2012 International Joint Conference on Neural Networks (IJCNN 2012), Brisbane, Australia, 2012.

11. M. Good, *MusicXML for Notation and Analysis*, in: *The Virtual Score: Representation, Retrieval, Restoration*, W. B. Hewlett and E. Selfridge-Field (ed.), The MIT Press, pp. 113-124, 2001.