IJCAI-97 Workshop on Issues in AI and Music - Evaluation and Assessment

Issues in Evaluating Beat Tracking Systems
Masataka Goto and Yoichi Muraoka
School of Science and Engineering, Waseda University
3-4-1 Ohkubo Shinjuku-ku, Tokyo 169, JAPAN.
{goto, muraoka}@muraoka.info.waseda.ac.jp

Abstract
This paper discusses the main issues in evaluating beat tracking systems and proposes amethod of evaluating the accuracy of these systems’ output. Although there have been a few
attempts to evaluate beat tracking systems,
they have not suﬃciently addressed quantitative evaluation of tracking accuracy, in particular for audio signals. Our method compares subjective hand-labeled beat positions
with computer-parsed beat positions and enables us to quantitatively analyze the deviation
error of the tracked rhythm (beats) not only at
the quarter-note level but also at the half-note
and measure levels. It also considers several
typical errors such as half-tempo and doubletempo errors as well as correct parsing. In
testing our beat tracking system for audio signals without drum-sounds on 40 popular songs
sampled from compact discs, we used the proposed method to evaluate overall recognition
rates and tracking quickness and accuracy.

1

Introduction

A great deal of research related to beat tracking has been
undertaken in recent years [Schloss, 1985; Dannenberg
and Mont-Reynaud, 1987; Desain and Honing, 1989;
1994; 1995; Desain, 1992; Katayose et al., 1989; Allenand Dannenberg, 1990; Driesse, 1991; Rosenthal, 1992a;
1992b; Rowe, 1993; Vercoe, 1994; Todd, 1994; Todd andLee, 1994; Large, 1995; Smith, 1996; Scheirer, 1996;
Goto and Abe, 1996; Goto and Muraoka, 1994; 1995a;
1995b; 1996; 1997] because beat tracking is an important initial step in the computational modeling of music
understanding. It is quite diﬃcult to understand Western music without perceiving beats, since the beat isa fundamental unit of the temporal structure of music.
Furthermore, beat tracking — also called beat induction, foot-tapping, or rhythm-tracking — has applications
in such various ﬁelds as human-computer improvisation,

video/audio editing, stage lighting control, and the synchronization of computer graphics with music.
Since various studies have addressed various issues,
purposes, and applications, it is generally diﬃcult tocomparatively evaluate previous beat tracking models.
There have, however, been a few discussions of evaluation issues. Desain and Honing [1995] described the evaluation of three diﬀerent rule-based beat-induction models from various points of view. Goto and Abe [1996]
also examined several cognitive models of rhythmic interpretation from the psychological viewpoint. Rosenthal [1992a] pointed out that evaluating the ideas behind many of previous systems is not a straightforwardtask and that diﬀerent researchers have worked on diﬀerent aspects of the problem. Because most studies havedealt with MIDI signals or used onset times as their input, the evaluation of audio-based beat tracking has notbeen discussed enough.
To take the ﬁrst step toward evaluating beat tracking
systems, we propose a method of measuring the accuracyof beat tracking results that enables those results to beevaluated quantitatively. This method compares subjective hand-labeled beat positions with computer-parsed
beat positions while considering diﬀerent rhythmic levels
and several typical errors as well as the correct parsing.
For example, it evaluates not only the quarter-note levelbut also the half-note and measure (bar) levels and also
analyzes half-tempo and double-tempo errors.
In the following sections we mainly describe issues
in the evaluation of beat tracking systems and introduce a measure of beat-tracking accuracy. Although theproposed evaluation method has been applied only tothe output of our audio-based beat-tracking system, we
think that it could also be applied to other systems for
MIDI signals or onset times.

2

Evaluation Issues

Since beat tracking systems can have a variety of inputs,
purposes, levels of rhythmic structure, and so on, various
issues should be taken into consideration when evaluating these systems. This section deals with some of the−9 −because such processing will facilitate future real-time
implementations.

important issues.

2.1

Inputs2.3

The input sources of beat tracking systems can be classiﬁed into three categories: onset times (a sequence ofinter-onset intervals), MIDI signals (a standard MIDI ﬁleor a real-time MIDI input), and audio signals. Withineach category there are the score-like uninﬂected inputs,
the real performances with roughly constant tempo, andthe expressive performances with tempo changes. In
terms of musical genres, previous systems have dealt
with classical works, folksongs, national anthems, popular music, rock music, and so on.
This variety causes researchers to address various aspects of the beat-tracking problem. That research all has
its own value; however it is generally diﬃcult to compare
the results of diﬀerent systems and to integrate the ideas
behind them. A ﬁrst step toward this comparison would
be a quantitative evaluation of the results of each system.

2.2

Rhythmic Structure

In this paper the rhythmic structure is deﬁned as a hierarchical musical structure with several levels of rhythm:
the quarter-note level, the half-note level, the measurelevel, and so on (Figure 1). The beginnings of a quarter
note, a half note, and a measure are respectively called abeat time, a half-note time, and a measure time. For thesake of brevity, we consider only the 4/4 time-signaturein this paper.
time

Measure level(measure times)

Half-note level(half-note times)

Quarter-note level(beat times)

Figure 1: Rhythmic structure.

Purposes

There are various purposes for tracking beats: understanding human rhythm perception, building a computational model understanding rhythm, following live performances, beat-quantization for automatic transcription, and synchronization of computer graphics with music. Beat-tracking researchers have accordingly speciﬁed
their problems in diﬀerent ways: ﬁnding beat positions
in onset times, following tempo changes of MIDI signals,
parsing the rhythmic structure of MIDI signals, and understanding the rhythmic structure of audio signals.
The criteria used to evaluate beat-tracking accuracy
depend on the purpose. For example, when the main
research purpose is to analyze tempo changes and expressiveness in human performances, the criteria should
be strict and the time-resolution should be ﬁne, since
the timing of beats is important. On the other hand,when the main application is music-synchronized computer graphics, the time-resolution of beat times is resampled at the frame-rate resolution (usually 30 frames
per second) in the output images. In such a case, thestability of tracking is more important than the precision of the beat times. In the case of beat tracking
for audio/video editing systems, even beat-tracking results that include typical errors such as half-tempo anddouble-tempo errors might be useful since human correction can be expected. An evaluation method whose
measurement results can be considered with regard to avariety of criteria is hence preferable.
Although real-timeness is not directly related to theevaluation of beat-tracking accuracy, it is an important
issue for several applications. Even if the preliminary implementations of a system do not work in real time, it isoften preferable to process the information sequentially

Most previous systems dealt with only a single level— usually the quarter-note level — which is the levelof regular isochronal pulses that a person feels in music. One system [Goto and Muraoka, 1994; 1995a; 1995b;
1996] also tracked beats at the half-note level, and othersystems [Rosenthal, 1992a; 1992b; Goto and Muraoka,
1997] tracked beats at the measure level as well as thelower levels. In evaluating those systems, it is necessary
to take these rhythmic levels into account.

3

Quantitative Measure of Accuracy

As an initial step toward the complete evaluation of beattracking systems, we propose a measure that can be utilized to analyze how accurate the tracked beat times
are. Although this is not a perfect measure considering
all evaluation issues, it is useful for analyzing the accuracy of beat-tracking results at several rhythmic levels
and identifying typical errors. Because we have beenworking on a real-time beat-tracking system for audio
signals [Goto and Muraoka, 1994; 1995a; 1995b; 1996;
1997], in this section we concentrate on describing themeasure for evaluating audio-based beat-tracking systems.

3.1

Beat Correctness

Since the beat is a perceptual concept that a person feels
in music, it is generally diﬃcult to deﬁne the correct beatin an objective way. We think that one of goals of beattracking research is to establish a precise deﬁnition of
the “beat” concept and to verify this deﬁnition using an
actual system.
In designing our measure for evaluation, we consideredsubjective hand-labeled beat positions to be the correct−10 −Figure 2: Beat-position editor program.
beat times. To label these positions, we developed abeat-position editor program that enables a user to markthe beat positions in a digitized audio signal while listening to the audio and watching its waveform (Figure 2).
The positions can be ﬁnely adjusted by playing back theaudio with click tones at beat times, and the user alsodeﬁnes a hierarchical rhythmic structure — the quarternote, half-note, and measure levels — corresponding to
the audio signal.

3.2

Designing Measure

The basic concept of our measure of beat-tracking accuracy is to compare the hand-labeled beat times (thecorrect times) with the beat times of the system output(the examined times)1 . In other words, we compare thesystem’s parsing with our own subjective parsing.
Basic Measure
We ﬁrst deﬁne the basic measure for analyzing the beattracking accuracies at the quarter-note, half-note, andmeasure levels. These accuracies are each represented
as a measurement set {Q, H, M}[τ, µ, σ, M ], where Q[],
H[], and M[] respectively represent the measures at thequarter-note, half-note, and measure levels. The term τis the correctly-tracked period (the period in which therhythm is tracked correctly), and the terms µ, σ, andM are respectively the mean, standard deviation, andmaximum of diﬀerence between the correct time and theexamined time.
The measurement set at each level is calculated asfollows:
1. Pairing the correct times with the examined times
Let Cn be the n-th correct time and Bm be the m-th
examined time. We ﬁrst ﬁnd the Bm which fulﬁlls
Condition (1) and is nearest to Cn in order to makea pair [Cn , Bm ]:1
1(1)
Cn − In−1 ≤ Bm < Cn + In2
2
1

In evaluating the quarter-note, half-note, and measurelevels, we use the beat, half-note, and measure times respectively.

−11 −In = Cn+1 − Cn(2)where In is the n-th inter-beat interval (the temporal diﬀerence between two successive beats). If
there is no Bm which fulﬁlls Condition (1), Cn islabeled as unpaired. An examined time Bm that is
not paired with a Cn is also labeled as unpaired.
2. Evaluating the error in each pairThe normalized diﬀerence (deviation error) Pn between Cn and Bm in each pair (Figure 3) is givenby |Bm −Cn |(Cn is paired)
in
(3)
Pn =
1
(Cn is unpaired)In / 2(Bm ≥ Cn )(4)
in =In−1 / 2 (Bm < Cn )
error

B m-1

Bm

B m+1

1
Pn

0

in
C n-1

Cn

time
C n+1

Figure 3: Evaluating the error.
3. Finding the longest correctly tracked period
The longest correctly tracked period τ = [CN s , CN e ]is then found in Cn according to Pn , where N s andN e are respectively the beginning and end suﬃx ofCn during the period. The correctly tracked period
should fulﬁll two conditions: every Pn during theperiod must be Pn < Φ where Φ (equal to 0.35 in ourevaluation) is a constant threshold, and there must
be no unpaired Bm during the period. In particular,[CN s , −] means that a beat tracking system keeps
on tracking the correct rhythm if once it starts totrack the correct one at CN s , and [!, !] means that asystem cannot track the correct rhythm at all.

4. Calculating the undecided terms of the measurement
set
Finally, the mean µ, standard deviation σ, and maximum M of Pn during the period τ are calculated.
Advanced Measure
We upgrade the basic measure to an advanced measurethat considers the typical errors: a half-tempo or doubletempo error and a π-phase error. The half-tempo anddouble-tempo errors are respectively those in which the
tracked inter-beat interval is twice or half the correct one.
The π-phase error is the half-period displacement error
in which the diﬀerence between the correct times andthe examined times is half of the interval between two
successive correct times. These kinds of errors should
to some extent be considered to be appropriate becausesuch rhythm-parsings are better than a random parsing.
The advanced measure of each rhythmic level is represented as a measurement set {Q, H, M}[τ, µ, σ, M, fT , fp ]based on the basic measure, where fT is a ﬂag indicating
the tempo error and fp is a ﬂag indicating the phase error. The ﬂag fT takes a value of − (the correct tempo),
hlf (the half-tempo error), or dbl (the double-tempo error), and the ﬂag fp takes a value of either 0 (the correct
phase) or π (the π-phase error). The fT of all the measurement sets {Q, H, M}[] takes the same value.
Correct times

Examined times

D-levelM-levelM-levelH-levelM-levelH-levelQ-levelH-levelQ-levelM-levelH-levelQ-levelE-levelQ-level

D: Double-measure
M: Measure[dbl, ]
[dbl, 0]

H: Half-note
Q: Quarter-note[-,][hlf, ]
[hlf, 0][-, 0]

E: Eighth-note[ fT , fp ]Figure 4: Advanced measure.
To determine fT , we ﬁrst evaluate six sets of the basic
measure Q[] between the examined times at the quarternote level and the six kinds of correct times at the threerhythmic levels with two phase variations: the eighthnote level with the 0- or π-phase (0-phase means thecorrect phase), the quarter-note level with the 0- or πphase, and the half-note level with the 0- or π-phase(Figure 4). We then ﬁnd the best-matched measure Q[]
with the longest correctly tracked period τ . The ﬂag
fT is thus determined on the basis of the level with its
best measure. The higher-level measurement sets {H,M}[] are calculated while considering the level of thedetermined Q[] to be the lowest (Figure 4).
To determine fp of each measurement set, we then calculate two sets of the basic measure at each level between

the examined times and the two kinds of correct times
at the corresponding level with the two phase variations.
The ﬂag fp is ﬁnally determined according to the phasewith the longer correctly tracked period τ . For example,
if the basic measure Q[] between the examined times
at the quarter-note level and the correct times at theeighth-note level with the π-phase has the longest τ , wedetermine Q[τ, µ, σ, M, dbl, π].
This advanced measure is useful for analyzing andidentifying the diﬀerent types of typical errors. Thismeasure also considers that one of disputable pointsof human rhythm parsing is the half-tempo or doubletempo error.

4

Experimental Results

With the advanced measure we evaluated the output ofour beat tracking system for monaural musical audio
signals without drum-sounds (a bass drum or a snaredrum). In this section we ﬁrst introduce an overview of
the target system and then describe recognition rates ofsongs, how quickly the system started to track the correct rhythm, and how accurately the system obtained
the beat, half-note, and measure times.

4.1

Target System for Evaluation

As mentioned above, the target system was our real-timebeat-tracking system for audio signals without drumsounds2 presented in [Goto and Muraoka, 1997]. Thissystem is an upgraded version of the system presented in[Goto and Muraoka, 1996]: this system can track beatsat the quarter-note, half-note, and measure levels. The
system assumes that the time signature of an input songis 4/4 and that its tempo is constrained to be between
61 M.M. (Mälzel’s Metronome: the number of quarter
notes per minute) and 120 M.M., and is roughly constant. Since the system digitizes the input at 22050 Hz
and the FFT window of the input is shifted by 256 samples, the time resolution of our measurement is 11.61 ms(= 256 / 22050 Hz).
The system is based on multiple-agent architecture inwhich multiple beat-position hypotheses are maintainedby programmatic agents using diﬀerent strategies forbeat tracking. This architecture enables the system tocope with diﬃcult beat-tracking situations: even if someagents lose track of beats, the system will track beatscorrectly as long as other agents maintain the correct
hypothesis. Each agent predicts beat times using autocorrelation and cross-correlation of the onset times inseven diﬀerent frequency ranges. The agent then recognizes the higher-level rhythmic structure and evaluates
the reliability of its own hypothesis. The ﬁnal output is2

We have also developed a real-time beat-tracking system
for audio signals that include drum-sounds [Goto and Muraoka, 1994; 1995a; 1995b].

−12 −Song that was incorrectly tracked
at the half-note or measure level
Song that was incorrectly tracked
at the quarter-note level

Overall Result

We tested the target system on 40 songs, each at least
about one minute long, performed by 28 artists. The
input audio signals were sampled from commercial compact discs of popular music and contained the sounds ofvarious instruments (not including drums). The timesignature was 4/4 and the tempi ranged from 62 M.M.
to 116 M.M. and were roughly constant.
We judged that a song was tracked correctly at a certain rhythmic level if the corresponding advanced measurement set of the song fulﬁlled {Q, H, M}[[CN s < 45.0
sec, CN e = −],3 µ < 0.2, σ < 0.2, M < 0.35, fT = −,fp = 0]. When the beat-tracking result for a song wasevaluated as Q[[10.84 sec, −], 0.072, 0.059, 0.243, −, 0],
H[[12.65 sec, −], 0.037, 0.029, 0.120, −, 0], and M[[25.05
sec, −], 0.019, 0.015, 0.046, −, 0], for example, we could
judge that this song was tracked correctly at the threerhythmic levels, that the deviation error was quite small
on the average, and that the maximum error was thethirty-second-note period at most.
In our experiment the system correctly tracked beatsat the quarter-note level in 35 of the 40 songs (87.5 %)and correctly recognized the half-note level in 34 of the
35 songs (97.1 %) in which the correct beat times wereobtained. Moreover, it correctly recognized the measurelevel in 32 of the 34 songs (94.1 %) in which the correct
half-note times were obtained.
We analyzed tracking errors at each of the three rhythmic levels using the advanced measurement sets. The
system did not track the correct beats at the quarternote level in ﬁve songs. In one of the ﬁve songs, thesystem made the π-phase error (fp of Q[] = π) during
most of the song (τ = [11.45 sec, −]). In the other foursongs, although the system temporarily tracked the correct beats, it often tracked wrong phases or determined
wrong inter-beat intervals. In the case of one song which
was incorrectly tracked at the half-note level, the system
had made the π-phase error (fp of H[] = π) just after
starting to track the correct beat times. There were two
3Song that was tracked correctly

The ﬁrst correct time C1 is considered 0 sec.

10
8
6
4
2
0
60

80

100

120

Tempo

Figure 5: Histogram of the tempo of all the songs.

4.3

Tracking Quickness

We can use the CN s of each measurement set to evaluate
tracking quickness — how quickly the system started to
track the correct rhythm — at each level. Figure 6 shows
the CN s of the correctly-tracked songs at the quarternote, half-note, and measure levels. The horizontal axis
represents the song numbers (#) arranged in order ofQuarter-note level (Song #1-35)
Half-note level(Song #1-34)
Measure level(Song #1-32)

Start time [sec]4.2

songs in which the system was unable to track only themeasure level. In those songs, the system also made theπ-phase error (fp of M[] = π).
Figure 5 shows the histogram of the correct tempo of
all the input songs. It also indicates the ﬁve songs whichwere incorrectly tracked at the quarter-note level andthe three songs which were incorrectly tracked at thehalf-note or measure level.

The number of songs

determined on the basis of the most reliable hypothesis
out of all these agent-generated hypotheses.
To make context-dependent decisions such as thoseneeded for recognizing the higher-level rhythmic structure and for determining which is the best hypothesisin an ambiguous situation, higher-level processing usingmusical knowledge is indispensable. The system therefore detects chord changes without chord name symbolization and utilizes heuristic musical knowledge ofchord changes. For example, by using its knowledge thatchords are more likely to change at the beginning of measures than at other positions, the system can determine
the higher-level structure.

45
40
35
30
25
20
15
10
5
0
1

5

10

15

20

25

30

35

Song number

Figure 6: Start time (CN s ) of tracking the correct
rhythm at the quarter-note, half-note, and measure levels.

−13 −CN s of the quarter-note level up to song #32. In eachsong where the rhythmic structure was eventually deter-

Table 1: Mean, minimum, and maximum of CN s at thequarter-note, half-note, and measure levels.
rhythmic level
mean
min
max
Quarter-note level 10.71 sec 0.79 sec 35.77 sec
Half-note level 14.70 sec 3.42 sec 42.56 sec
Measure level 20.70 sec 3.42 sec 42.56 sec

Deviation error

Mean
Maximum
0.4

0.3

0.2

mined correctly, the system initially had trouble determining the higher rhythmic level, even though the lower
level was correct. The mean, minimum, and maximumof the CN s of all the correctly-tracked songs are shownin Table 1.

4.4

Tracking Accuracy

We can use the µ, σ, and M of each measurement set
to evaluate tracking accuracy — how accurate the examined times (the system output) were — at each level.
Figure 7 shows the µ and M of the correctly-tracked
songs at the three rhythmic levels. The horizontal axis
represents the song numbers rearranged in order of µ of
the quarter-note level up to song #32. The mean, minimum, and maximum of µ and M of all the correctlytracked songs are shown in Table 2 and Table 3, respectively. The maximum deviation error was 0.339 and themean of the error was relatively small on the average.

0.1

0.0
1

5

10

15

20

25

30

35

Song number(a) Quarter-note level (Song #1-35)

Deviation error

Mean
Maximum

Table 2: Mean, minimum, and maximum of µ at thequarter-note, half-note, and measure levels.
rhythmic levelmean minmax
Quarter-note level 0.062 0.019 0.130
Half-note level 0.031 0.009 0.069
Measure level 0.015 0.005 0.036

0.2

Table 3: Mean, minimum, and maximum of M at thequarter-note, half-note, and measure levels.
rhythmic levelmean minmax
Quarter-note level 0.223 0.053 0.339
Half-note level 0.101 0.026 0.174
Measure level 0.045 0.013 0.077

0.15

0.1

0.05

0.0
1

5

10

15

20

25

30

35

Song number

5(b) Half-note level (Song #1-34)

Deviation error

Mean
Maximum
0.1

0.05

0.0
1

5

10

15

20

25

30

35

Song number(c) Measure level (Song #1-32)
Figure 7: Mean (µ) and maximum (M ) of the deviation
error at the quarter-note, half-note, and measure levels.

Measuring Rhythmic Diﬃculty

Although the proposed measuring method enables us toquantitatively analyze the output of beat tracking systems, there are lots of other evaluation issues that wehave not discussed so far. It is, for example, importantbut very diﬃcult to measure the rhythmic diﬃculty ofa song from the viewpoint of the input of beat tracking.
For the input we prefer realistic songs such as ones sampled from commercial compact discs rather than artiﬁcial songs generated just for experiments. On the other
hand, one reasonable solution might be to carefully compose songs with diﬀerent levels of diﬃculty.
To take the ﬁrst step toward measuring the rhythmic diﬃculty of realistic songs, we tried to evaluate only
the power transition of the input audio signals, although
it is desirable to consider various aspects of songs suchas tempo changes, rhythmic patterns (complexity), time
signatures, musical instruments, and musical genres. In−14 −1. Finding the local maximum of the power
Let a(t) be the power of the input audio signal attime t,4 which is the sum of the absolute value ofthe digitized audio data during the corresponding
period. We ﬁrst calculate two kinds of local maximum of the power, Lbn and Lon :
Lbn

=

Lon

=

max

(a(t))(5)max(a(t))(6)Cn −  ≤ t < Cn + In /4 − 

Cn + In /4 −  ≤ t < Cn+1 − 

where Cn is the n-th correct beat time of thequarter-note level and In is the n-th inter-beat interval (Figure 8).  is a margin to obtain appropriate maximum values because the power of sounds
played on the beat sometimes increases just before
its beat time. In our current implementation,  isequal to 2 (23.22 ms). Thus the Lbn represents the
maximum power on the n-th beat and the Lon represents the maximum power on the other positions
between the n-th beat and (n + 1)-th beat.
power

L

b
n-1

L

o
n-1

L

b
n

L

o
n

We ﬁnally calculate the power-diﬀerence measure
D, which is the mean of all the dn in the song:
D=

N−1
1 
dn
N − 1 n=1(8)where N is the number of the correct beat times.
The D takes a value between 0.0 (easiest) and 1.0(most diﬃcult). A regular pulse sequence with aconstant interval, for example, takes 0.0 of this measure. Practically speaking, the D of a realistic songcannot take 1.0.
With the power-diﬀerence measure D, we evaluatedthe rhythmic diﬃculty of all 40 songs that we used fortesting the beat tracking system described in Section 4.
Figure 9 shows the histogram of D of all the songs.
It also indicates the ﬁve songs which were incorrectly
tracked at the quarter-note level and the three songs
which were incorrectly tracked at the half-note or measure level. Although this measure is not perfect for evaluating the rhythmic diﬃculty, it showed a tendency toagree with our subjective ordering of the rhythmic diﬃculty of the songs.
Song that was tracked correctlySong that was incorrectly tracked
at the half-note or measure level
Song that was incorrectly tracked
at the quarter-note level

The number of songs

terms of the power transition of the audio signals, it ismore diﬃcult to track beats of a song in which the power
on beats is often less than on other positions between adjacent beats. In other words, the larger the number ofsyncopations, the greater the diﬃculty of tracking beats.
We therefore consider diﬀerences between the power
on beats and the power on other positions as a measure of
the rhythmic diﬃculty. This measure, called the powerdiﬀerence measure, is calculated as follows:

10
8
6
4
2
0

a(t)

0.4

0.45

0.5

0.55

0.6

Power-difference measure

I n-1
C n-1

In
Cn

Figure 9: Histogram of the power-diﬀerence measure of
all the songs.

time
C n+1

Figure 8: Finding the local maximum of the power.

6

2. Calculating the normalized power diﬀerence of eachbeat
The normalized power diﬀerence dn between Lon and
Lbn is then calculated:
dn = 0.5

Lon − Lbn
+ 0.5
max (Lon , Lbn )(7)

3. Calculating the mean of the normalized power differences in a song
4
The time resolution of this measurement is 11.61 ms (=256 / 22050 Hz) in our current implementation.

Conclusion

We have discussed the main issues in evaluating beattracking systems and have addressed several of them byproposing a quantitative measure that analyzes the accuracy of a system’s rhythm-parsing. Although this is not
perfect measure addressing all evaluation issues, it cananalyze the accuracy of the rhythm tracking while considering a hierarchical rhythmic structure — the quarternote, half-note, and measure levels — and several typicalerrors such as the half-tempo and double-tempo errors
and the π-phase error. In our experiment the proposed
measure was applied and found to be useful for evaluating the recognition rates and the tracking quickness andaccuracy of our audio-based beat-tracking system.

−15 −We plan to address other evaluation issues and to consider an advanced framework of beat-tracking researchevaluation. Future work will also include upgrading ourbeat tracking system so that it can understand the rhythmic structure of music in a human-like fashion.

References[Allen and Dannenberg, 1990]
Paul E. Allen and Roger B. Dannenberg. Tracking
musical beats in real time. In Proc. of the 1990 Intl.
Computer Music Conf., pages 140–143, 1990.
[Dannenberg and Mont-Reynaud, 1987] Roger B. Dannenberg and Bernard Mont-Reynaud. Following an
improvisation in real time. In Proc. of the 1987 Intl.
Computer Music Conf., pages 241–248, 1987.
[Desain and Honing, 1989] Peter Desain and Henkjan
Honing. The quantization of musical time: A connectionist approach. Computer Music Journal, 13(3):56–
66, 1989.
[Desain and Honing, 1994] Peter Desain and Henkjan
Honing. Advanced issues in beat induction modeling:
syncopation, tempo and timing. In Proc. of the 1994
Intl. Computer Music Conf., pages 92–94, 1994.
[Desain and Honing, 1995] Peter Desain and Henkjan
Honing. Computational models of beat induction:
the rule-based approach. In Working Notes of the
IJCAI-95 Workshop on Artiﬁcial Intelligence and Music, pages 1–10, 1995.
[Desain, 1992] Peter Desain. Can computer music beneﬁt from cognitive models of rhythm perception? In
Proc. of the 1992 Intl. Computer Music Conf., pages
42–45, 1992.
[Driesse, 1991] Anthonie Driesse.
Real-time tempo
tracking using rules to analyze rhythmic qualities. In
Proc. of the 1991 Intl. Computer Music Conf., pages
578–581, 1991.
[Goto and Abe, 1996] Yasuhiro Goto and Junichi Abe.
On cognitive models of rhythmic interpretation. International Journal of Psychology, 31:51, 1996.
[Goto and Muraoka, 1994] Masataka Goto and Yoichi
Muraoka. A beat tracking system for acoustic signals of music. In Proc. of the Second ACM Intl. Conf.
on Multimedia, pages 365–372, 1994.
[Goto and Muraoka, 1995a] Masataka Goto and Yoichi
Muraoka. Music understanding at the beat level —real-time beat tracking for audio signals —. In Working Notes of the IJCAI-95 Workshop on Computational Auditory Scene Analysis, pages 68–75, 1995.
[Goto and Muraoka, 1995b] Masataka Goto and Yoichi
Muraoka. A real-time beat tracking system for audio
signals. In Proc. of the 1995 Intl. Computer Music
Conf., pages 171–174, 1995.

[Goto and Muraoka, 1996] Masataka Goto and Yoichi
Muraoka. Beat tracking based on multiple-agent architecture — a real-time beat tracking system for audio signals —. In Proc. of the Second Intl. Conf. on
Multiagent Systems, pages 103–110, 1996.
[Goto and Muraoka, 1997] Masataka Goto and Yoichi
Muraoka. Real-time rhythm tracking for drumless audio signals — chord change detection for musical decisions —. In Working Notes of the IJCAI-97 Workshop
on Computational Auditory Scene Analysis, 1997 (inpress).
[Katayose et al., 1989] H. Katayose, H. Kato, M. Imai,and S. Inokuchi. An approach to an artiﬁcial musicexpert. In Proc. of the 1989 Intl. Computer Music
Conf., pages 139–146, 1989.
[Large, 1995] Edward W. Large. Beat tracking with anonlinear oscillator. In Working Notes of the IJCAI95 Workshop on Artiﬁcial Intelligence and Music,
pages 24–31, 1995.
[Rosenthal, 1992a] David Rosenthal. Emulation of human rhythm perception. Computer Music Journal,
16(1):64–76, 1992.
[Rosenthal, 1992b] David Rosenthal. Machine Rhythm:
Computer Emulation of Human Rhythm Perception.
PhD thesis, Massachusetts Institute of Technology,
1992.
[Rowe, 1993] Robert Rowe. Interactive Music Systems.
The MIT Press, 1993.
[Scheirer, 1996] Eric D. Scheirer. Using bandpass andcomb ﬁlters to beat-track digital audio. (unpublished),
1996.
[Schloss, 1985] W. Andrew Schloss. On The AutomaticTranscription of Percussive Music — From Acoustic
Signal to High-Level Analysis. PhD thesis, CCRMA,
Stanford University, 1985.
[Smith, 1996] Leigh M. Smith. Modelling rhythm perception by continuous time-frequency analysis. In
Proc. of the 1996 Intl. Computer Music Conf., pages
392–395, 1996.
[Todd and Lee, 1994] Neil Todd and Chris Lee. An
auditory-motor model of beat induction. In Proc. of
the 1994 Intl. Computer Music Conf., pages 88–89,
1994.
[Todd, 1994] Neil P. McAngus Todd. The auditory “primal sketch”: A multiscale model of rhythmic grouping. Journal of New Music Research, 23(1):25–70,
1994.
[Vercoe, 1994] Barry Vercoe. Perceptually-based music
pattern recognition and response. In Proc. of the Third
Intl. Conf. for the Perception and Cognition of Music,
pages 59–60, 1994.

−16 −