Issues in Tracking Musical Beats

Beat Tracking based on
Multiple-agent Architecture

Peak-finding with a threshold is not sufficient
Many energy peaks are not directly related to beats
Multiple interpretations of beats are possible

A Real-time Beat Tracking System
for Audio Signals

No specific sound directly indicates the beat positioninter-beat interval

Ambiguous situations

Several events may correspond to a beat
Different inter-beat intervals seem plausible

School of Science and Engineering, Waseda University

Masataka Goto

Context-dependent decisions using musical knowledge

Yoichi Muraoka

Whether a beat is strong or weak
Which is the best interpretation

1996/12/11 ICMAS 96
No.1

No.5

1. Introduction

Our Solutions
6k-11k2k-6k
1k-2k
500-1k
250-500
125-2500-125Hz

Detect multiple tracking cues

Application of Multiple-agent Architecture

Onset times in several frequency ranges

Application to beat tracking for musical audio signals

Chord change possibility

Understand real-world audio signals
Need to handle various ambiguous situations

Maintain multiple hypotheses

Advantages
Interpret input signals in various ways

Each corresponds to a hypothetical interpretation

Multiple agents can examine multiple hypotheses
according to different strategies
Main contribution
Multiple-agent architecture is actually usefulfor a practical real-world application
No.2time

No.6

2. Multiple-agent Architecture

What is Beat Tracking?
Track quarter notes
just as people keep time to music by foot-tapping

Multiple Agents Examine Multiple Hypotheses
Agents track beats according to different strategies

Why is Acoustic Beat-tracking Important?

Each maintains a beat-position hypothesis:
Next beat time

Computational model of human music perception
Basic unit of the temporal structure of music

Beat typeInter-beat interval
predict

inter-beat interval

time

Agent 1

Useful in various applications

Agent 2

Video/Audio editing systems

Agent 3

Stage lighting control

predicted
next-beat time

Agent 4

Music-synchronized CG animation

Agent 5

No.3

No.7

Beat Tracking Problem

Definition of "Agent"

Organize music into almost regularly spaced beats

1. The agent interacts with other agentsto perform a given task

Musical audio signals(sounds of various
kinds of instruments)

2. The agent evaluates its own behavior
on the basis of the input

time
Beat times(quarter-note level)

60%

Beat type(half-note level)

Assumptions
No.4

Strong Weak Strong Weak Strong Weak??

20%90%

3. The agent adapts to the input
by adjusting its own behavior

44
Tempo : 61-120 M.M. almost constantTime signature :No.8

3. System Description(for music without drum-sounds)

Evaluation

60%??

20%

90%

2. Each agent evaluates its own hypothesis
according to the input acoustic signals

Model
Frequency spectrum
Onset components

12 Higher-level
checkers

Compact disc

Reliability of the hypothesis
Quantitative result of the self-evaluationft

Evaluate using musical knowledge
Beat information
ManagerMusicalaudio signals
7 Onset-time finders
3 Onset-time vectorizers
FrequencyAnalysis

A/D Conversion

Manager decides which is the best hypothesis
according to the reliability?

12 Agents

20%?

Beat Information
Transmission

Beat Prediction

No.9

90%

60%
No.13

Make a Hypothesis
Onset-time vector

Evaluation(Result of Frequency Analysis)

Two kinds of musical knowledge

6k-11k2k-6k
1k-2k
500-1k
250-500
125-2500-125
Hz

1. Sounds are likely to occur on beats
Correct beat times tend to coincide with onset times
2. Chords are more likely to change on beats
than on other positions
between two successive correct beats

time

Autocorrelation

Inter-beat interval

Cross-correlation

Prediction field

Chord changes

Beat type

Beat times

Agent-generated hypothesis
Chord change
possibility

Beat time
Beat typetime

evaluate
how coincide

extrapolate

eighth-note displacement positions

Strong Weak Strong Weak Strong Weak

No.10

No.14

Interaction

Evaluation

1. Each agent interacts with another agent
to perform the beat-tracking task

Manager
1. Classify all hypotheses into groups

12 agents are grouped into 6 pairs

according to beat time and inter-beat interval

Paired agents predict next beat times cooperatively

2. Calculate overall reliability of each groupexamine the same inter-beat interval

3. Select the dominant group
4. Repeat three times while narrowing
the allowable margin of beat times

Agent pair 1

Agent pair 2

time

the reliable hypothesisin the most dominant group

Compensate for one of typical tracking errors
No.11

No.15

Interaction

predict

Adaptation

Interaction through prediction field

3. Each agent adapts to the current input
by adjusting its own strategy parameter

Expectancy curve:
when the next beat is expected to occur

The reliability becomes high enoughHeight of each local peak:
the next beat-position possibility

Tune a parameter to narrow
the range of possible inter-beat intervals

Paired agents inhibit each other’s fieldExamine only a neighborhood of
the current appropriate inter-beat interval

prediction
field

986ms

inter-beat interval

Agent 1-1
pairinhibit
now

current
intervalpossible range

500ms

Keep the good hypothesis that has
the inter-beat interval appropriate to the input

Agent 1-2
No.12

narrowed rangetime
No.16

4. Experimental Results
Conditions(Implemented on AP1000)

Audio signals without drum-sounds from CDs
40 popular songs performed by 28 artists
Tempo: 62-116 M.M.

roughly constant

Results
Correctly tracked beats in 34 out of 40 songs
Mistakes in 6 songs
beat times were wrong (4)very few onset times
temporarily fluctuated tempo

beat type was wrong (2)irregularity of chord changes

No.17

5. Conclusion
Summary
Multiple-agent architecture for beat tracking
Examine multiple hypotheses in parallel
Each agent is capable ofinteractioncompensate for typical error

self-evaluation

decide which is the best hypothesis

adaptation

keep the good hypothesis

Robust enough to handle real-world audio signals
No.18

Future Work
Upgrade our beat-tracking system
to make use of other higher-level musical structure
More sophisticated interaction among agents
Application of the multiple-agent architecture
to other perceptual problems

No.19