Isao Hidaka, Masataka Goto, and Yoichi Muraoka: An Automatic Jazz Accompaniment System Reacting to Solo,
Proceedings of the 1995 International Computer Music Conference, pp.167-170, September 1995.

An Automatic Jazz Accompaniment System Reacting to Solo
Isao Hidaka Masataka Goto Yoichi Muraoka
School of Science and Engineering , Waseda University
3-4-1 Ohkubo Shinjuku-ku, Tokyo 169, JAPAN.
+81-3-3209-5198/fhidaka,goto,muraokag@muraoka.info.waseda.ac.jp
ABSTRACT: This paper presents an automatic accompaniment system for jazz that understands solist’s
intention of improvising and outputs expressive accompaniment of a bass guitar and drums reacting to solo.
Previous systems extracted just one parameter as solist’s intention and output accompaniment by switchingregistered patterns. Our system can understand solist’s intention in more detail by extracting five parameters
as his intention from solo. In addition, to make accompaniment rich in variation, this system directly alters
accompaniment data such as loudness, pitch and the number of notes, according to those parameters. In ourexperiment, it outputs appropriate accompaniment following the solist’s intention.

1 Introduction
This research aims to simulate human music listening process in a jazz ensemble, and cooperation amongplayers. Although there are various kinds of jazz, we consider only four-beat jazz standards with a constanttempo. We deal with trio play that consists of a human pianist (or a guitarist), a bassist and a drummer
by a computer. In this study we assume that the most significant player is a pianist, and the other two areaccompanists 0 i.e., the pianist improvises a solo with a certain intention, which plays very important rolein this ensemble. If the accompanists do not understand it, the ensemble sounds poor totally. It is therefore
desirable that the accompanists play following the solist’s intention.
A previous system called JASPER [Wake et al., 1994] extracted just one parameter called tension parameter as solist’s intention and output the accompaniment by switching pre-registered patterns according tothis parameter. It was not enough to express his intention with only this parameter, however. In anothersystem, Tsutomu Kanamori [Kanamori et al., 1993] implemented a listening model in a jazz session. They
proposed detecting four kinds of sensuous information from the performance as a message that represented
a psychological state of a player. In other related work, Robert Rowe implemented an interactive musicsystem called Cypher [Rowe, 1992]. This system classified MIDI input into six features such as loudness,duration, and density, to understand a human’s play. However, it didn’t extract player’s intention.
Our system can understand solist’s intention in more detail by extracting multiple parameters as hisintention from the various aspects of the solo. In addition, to make the accompaniment rich in variation,
this system alters the accompaniment data such as loudness, pitch, and the number of notes, according tothese parameters. This system takes MIDI data as input, plays based on stored data such as keynote, chord
progression, theme and plain accompaniment data, and outputs MIDI data as the accompaniment.
Our system extracts five parameters called intention parameters as solist’s intention from solo performance. In our current implementation, the intention parameters are excitement, tension, emphasis on chord,
chord substitution, and theme reprise. Since it is hard to extract these parameters from individual notes
directly, the system first detects eight musical primitives from the notes. These are louder note, chordtone, higher note, and so on. Each intention parameter is induced by summing up the particular musicalprimitives. Our system then alters the stored accompaniment data according to the intention parameters.
The intention parameters control two modules that directly alter the accompaniment data. One module
called MIDI filter adjusts loudness and pitch. The other called MIDI phase shifter adds notes by shifting
timing of notes.
This system has been implemented on a personal computer and works in real time. It was tested bya pianist and a guitarist. Experimental results show that it was able to extract intention parameters fromindividual notes of solo, and output appropriate accompaniment of a bass guitar and drums that followed
the solist’s intention of improvising.

2 Features of our system
2.1

Multiple solist’s intention

Extracting various solist’s intention enables the system to understand the solo and to react to it appropriately.
We therefore extract solist’s intentions that are important for jazz play as five intention parameters fromvarious aspects of solo (Figure 1). The five intention parameters are as follow:

 excitement
The parameter excitement expresses a solist’s intention to make the ensemble inspiring. This intention
is significant in jazz as well as in other music genres. This parameter is similar to JASPER’s parameter.
The accompanists excite their play reacting to this intention.
 tension
The parameter tension expresses a solist’s intention to make the ensemble a little different from astandard by playing some notes such as tension notes and higher notes. Since tension notes are non-chord
notes, they cause a variety of chord feelings. For example, when our system detects many tension notes,
this intention is extracted.
 emphasis on chord
The parameter emphasis on chord expresses a solist’s intention to emphasize a standard’s chord
progression by playing a phrase that includes some chord notes specified in a song. For example, oursystem extracts this parameter when the solist plays a phrase such as arpeggios of the chord notes.
 chord substitution
The parameter chord substitution expresses a solist’s intention to emphasize standard’s chord progression
with slightly different from a song by playing a phrase that includes some substituted chord notes. Because
a substituted chord has the same function as the original chord has, the solist can play a phrase with some
substituted chord notes with keeping a feeling of the original chord.
 theme reprise
The parameter theme reprise expresses a solist’s intention to plays a standard’s theme or a faking
theme that can be recognized as the standard. When the system detects this intention, it outputs plain
accompaniment data without any alteration.

2.2

Dynamic alteration of accompanimentMusical Primitives

Intention Paramters

Alterationof accompaniment

louder note

To make a total ensemble better, it is necessary for theaccompanists to play reacting to the solist. We dynamically alters accompaniment data such as loudness, pitch
and the number of notes, according to the intention parameters mentioned above. Even if the solist suddenly
changes his play, the system can make accompaniment
more appropriate to solo than outputting by switching
pre-registered patterns.
Figure 1 shows five intention parameters, musicalprimitives that induce intention parameters, and alteration of accompaniment reacting to intention parameters. In the next section, we describe how the system
works in more detail.

3 System description

louder notehigher notemany notes

cymbals

excitement

many notes
tension notehigher note
tension notehigher note
tensiontension note

chord note
tension note

chord noteemphasis on chordnote in a scalenote in a scaleother chord notetheme note

tension note

chord substitution

theme reprise

other chord noteplain accompanimentendingFigure 1: Solist’s intention

Figure 2 shows an overview of our system. The system consists of three main stages, Detection of musicalprimitives, Extraction of intention parameters, Alteration of accompaniment. For the preparation of areal-time performance, chord and key are analized in the Analysis of chord and key stage, in advance. In
the Detection of musical primitives, eight musical primitives are obtained from the solo’s individual notes
using the results of analysis of chord progression and key. In the Extraction of intention parameters, fiveintention parameters are induced by summing up the corresponding musical primitives. In the Alterationof accompaniment, accompaniment data are directly altered according to the intention parameters by using
two modification modules.

3.1

Analysis of chord and key

The system analyzes the chord and key on the basis of music theory [Taniguchi, 1991] in order to understandchord function, tension notes in the stored chord progression, available note scale and so on. The systemutilizes these results to detect the musical primitives from solo performance and to alter accompanimentdata.

live solo by piano or guitar
Detection of musical primitives

Analysis of chord and key

matching

analyzing the chord and key

matching

Extraction of Intention Parameters
past primitives

summing up

chord progression
key
theme melody

Alteration of accompanimentaccompaniment data

modulemoduleaccompaniment of bass and drums

Figure 2: Overview of our system

3.2

Detection of musical primitives

For the preparation of extracting the intention parameters, the system detects eight musical primitives fromevery note to understand some of musical features of the notes. The musical primitives tension note, chordnote, scale note, and other chord note are detected by using the results of analysis of the chord and key.
Another primitive theme note is detected by matching the input note with the theme melody. This stage doesnothing for the other primitives louder note, higher note, and many notes, since they are simple primitives,and can be obtained directly.

3.3

Extraction of intention parametersmusicalprimitives
louder notehigher notemany notes
tension note
chord note
note in a scaleother chord notetheme note

exciteIntention parametersemphasischord subon chordstitution------70----------20----0------0----0-------tension

themereprise
In this stage, five intention parameters are ex100--tracted from musical primitives. Each intention
80--5--parameter is induced by summing up the par3--ticular musical primitives shown in Figure 1.
----To follow a solist correctly when he suddenly--------changes his intention, this system extracts these--3parameters on every beat, and to extract a slightbut continuous intention, this system sums up Table 1: Threshold values for extracting intention parameters
the musical primitives in a certain period.
Our system weights some musical primitives according to their importance in the intention parameters
under consideration. In summing up the primitives, our system ignores the primitives whose values are lessthan their threshold value. These threshold values are listed in Table 1. In this table, the value at the louder
note is MIDI velocity, and the values at the higher note are MIDI note number. The mark 3 indicates that theparameter theme reprise is detected on the basis of how the input notes are matched with the stored theme
without threshold.

3.4

Alteration of accompaniment

In this stage, our system dynamically alters the accompaniment data according to values of intention
parameters and the period for which the intention parameters extracted. To follow only the solist’s apparentintentions, the system alters accompaniment data when an intention parameter’s value is greater than it’sthreshold value.
The intention parameters control two modules that alter accompaniment data directly. One module called
MIDI filter adjusts loudness and pitch. The other called MIDI phase shifter puts additional notes by shifting
timing of the original notes. In this case, the intention parameters control how long and how many times

it shifts data. For example, when the intention parameter excitement is extracted, the system makes theaccompaniment note louder and higher with MIDI filter and it adds some notes with MIDI phase shifter. In
the case of altering the bass part, the pitch of notes is calculated on the basic of the results if the analysis of
the chord and key. In the case of altering the drums part, notes of other percussion instruments are added,or the part of a percussion instrument in the stored accompaniment is replaced by another instrument.

4 Experimental results

excitement

This system has been implemented on a personal com020406080100time[sec.]
puter (NEC PC-9821 Ap2) and works in real time. It
drums: added notes
was tested by a pianist who had studied music in thedrums: added cymbals
bass: added notes
university, and a guitarist who had experience of playtension
ing in an amateur jazz band. They played with a MIDI
synthesizer (KORG 01/W ) or a guitar that can output
020406080
100time[sec.]
bass: higher notes
MIDI streams. The accompaniment sounds are output
bass: tension notesby the synthesiser. We handled a standard called ‘‘takeemphasis on chord
the A train.’’ We requested them to play the theme in
020406080
100time[sec.]
the first half and then improvise.
bass: chord notes
Figure 3 shows how our system extracted five inchord substitution
tention parameters, and how it altered accompaniment
data according to these parameters. Both players evaltime[sec.]
020406080
100
bass: scale notes
uated that the system output the accompaniment at antheme repriseamateur level, and that it could nearly extract theirintentions. The system sometimes extracted several in020406080100time[sec.]
plain accompanimenttentions when the solist played with only one intention.
The reason for the mistakes is that a certain note caused
Figure 3: Intention parameters and accompaniment
the multiple musical primitives to be obtained.
One of the solists pointed out that the system reacted too sensitive. This was chiefly because we fixed
the threshold values that affected the extraction of intention parameters. We think that the system will react
more appropriate to solo by dynamically modifying these values or by changing them so that they fit thesolist.

5 Conclusion
In this paper, we proposed the way of extracting five parameters that express solist’s intention and accompanying him appropriately by altering the accompaniment data according to them. To extract parameters asintention, the system summed up several musical primitives, and to alter the accompaniment data directly,
we implemented two modules that handled raw MIDI data. Experimental results showed that our system
was able to extract the intention parameters and output appropriate accompaniment reacting to them.
We implemented a fixed accompanist because we statically defined all the threshold values concerningwith extraction of intentions. We plan to improve the algorithm for modifying those values automaticallyduring the performance. Future work will include the implementation of changing the accompanimentactively, and understanding solist’s feelings.

References[Wake et al., 1994] Sanae Wake, Hiroaki Katou, Naoki Saiwaki, Seiji Inokuchi: Cooperative Musical Partner System
Using Tension Parameter: JASPER (Jam Session Partner) Trans. of Information of Processing Society of Japan,
Vol. 35 No. 7 pp. 1469-1481, 1994 (in Japanese).
[Kanamori et al., 1993] Tsutomu Kanamori, Haruhiro Katayose, Seiji Inokuchi, Hiroshi Hirai, Yasuhisa Niimi: Interpretation of Musicality in Jazz Improvisation using Multi-Agent Model, Proceedings of IAKTA/LIST International
Workshop on Knowledge Technology in the Arts, pp.107-114, 1993.
[Kanamori et al., 1995] Tsutomu Kanamori, Haruhiro Katayose, Yasuhisa Niimi, Hiroshi Hirai, Seiji Inokuchi: AStudy of Music Recognition in a Jazz Session System, Trans. of Information of Processing Society of Japan, Vol.
36 No. 1, pp. 139-152, 1995 (in Japanese).
[Rowe, 1992] Robert Rowe: Machine Listening and Composing with Cypher, Computer Music Journal, Vol. 16, No.
1, pp. 43-63, 1992.
[Taniguchi, 1991] Hiroshi Taniguchi: The Real Jazz Guitar Volume 1, Tokyo Ongaku Syoin, pp. 15, 28-34, 57-70 1991
(in Japanese).