BTSMusical Acoustic Signals

44AP1000

Sounds of various kinds of instrumentsthat include drums

Time signature :Tempo : 70-180 M.M. almost constant

BTS
BTS

Correspond to quarter notes

W

1. Introduction

1994/10/20 ACM MULTIMEDIA 94

No.1

Beat Information

SYoichi Muraoka

Beat Time, Beat Type, Current Tempo

W

Masataka Goto

Transmitted in time to input musicS

School of Science and Engineering, Waseda UniversitySD
BD

Beat Information

A Beat Tracking System
for Acoustic Signals of Music

No.5

2. Acoustic Beat-Tracking Issues

Outline1. Introduction

Issues of Signal Processing

2. Issues and Overview

Issues of Real-time Processing

3. Implementation and Results

Issues of Musical Judgement

4. Applications
5. Conclusion
2. Acoustic Beat-Tracking Issues

No.2

No.6

1. Introduction

Issues of Signal Processing
1. No specific sound directly indicates the position of beats

What is Beat Tracking?
track quarter notes

Manage Multiple Evidence

just as people keep time to music by foot-tapping
Use onset times in several different frequency ranges

Multimedia Applications

Use onset times of two different instruments(Bass drum and Snare drum)Video editing systems
Use onset loudness in addition to time

Audio editing systems
Stage lighting control
Real-time computer graphics
1. Introduction

No.3

2. Acoustic Beat-Tracking Issues

No.7

Issues of Signal Processing

Previous Systems

2. Difficult to obtain precise onset times
from complicated acoustic signals

MIDI-input systems
MIDI

Non-real-time audio-input systems
Sophisticated Means of Estimating Onset TimeClassical repertoire
Audio

Multiple interpretations of onset times

Our System

Each assigned reliability

Real-time audio-input system
Complex sonic environment

AudioPopular music
1. Introduction

No.42. Acoustic Beat-Tracking Issues

No.8Beat Prediction＋＋＋＋＋＋

time

predict

30 Agents predict the Next Beat in Parallel＋＋

Maintain hypotheses
according to different strategies＋

Evaluate reliability＋

Select the most reliable hypothesis

onset times＋

beat times

pair＋

＋

2. Acoustic Beat-Tracking Issues

No.9＋

predict＋

inter-beat-interval＋

beat times＋(obtained previously)

＋

time

onset times＋

Predict the Next Beat Time in Advancepair

3. Only acoustic signals obtained previously are available
to track beats in real timeGrouped into 15 pairs

2 agents in same pair predict cooperatively

Issues of Real-time Processing

No.13

Musical Acoustic Signals

A/D Conversion
Sun SS10BI Transmission
Ethernet

1 cellBI Generation

1 cell30 cells

Managing Hypotheses

AgentsBeat Prediction

Input and Output process via SCSI

5 cells

Fast Fourier Transform

8 cells

Extracting onset components

Finding onset times

15 cells

Detecting BD and SD

1 cell

2. Acoustic Beat-Tracking Issues

No.10

4. Implementation

Generate BI on the basis of the most reliable hypothesis

Fujitsu AP1000

Each agent interprets onset times and makes hypothesis

Frequency Analysis

Manage multiple agents that track beats
according to different strategies

Distributed memory parallel computer AP1000

Maintain Multiple Parallel Hypotheses

64 cells are divided into 8 groups

Multiple parallelizing techniques

4. BTS should be able to recover from tracking errors

4. Implementation

Issues of Real-time Processing

No.14

Issues of Musical Judgement

System Configuration

5. Difficult to determine whether a beat is strong or weak

BTSBTS

Utilize Onset Times of Drum-Sounds
AP1000

Output Graphics

Parallel Computer
AP1000

1Bass Drum (BD) on Strong beatSnare Drum (SD) on Weak beat234SDBD

Workstations

EthernetCDSPARC Station 10

Particular BD or SD may be missing (e.g., Syncopation)
Automatically acquire characteristic frequencies

Output Sounds
Drums
Clapping

MIDI Instrument

2. Acoustic Beat-Tracking Issues

No.11

Musical Acoustic Signals

A/D Conversion

Fast Fourier Transform

5. Test Results
Conditions
30 popular songs
Monaural acoustic signal sampled from CD
BD and SD usually on the strong and weak beats

Results

Frequency
Analysis

Detecting BD and SD

onset times of BD and SDand reliability

Beat
Prediction

AgentsExtracting onset components

Finding onset times

3. Overview of BTS

onset times
and reliability

hypotheses

Managing Hypotheses

3. Overview of BTS

most reliable hypothesis

BI GenerationBI Transmission

Beat Information

No.12

4. Implementation

No.15

Correctly tracked beats in 27 out of 30 songs in real time
Efficacy of examining multiple hypotheses was confirmed
5. Experiments and Results

No.16

6. Applications of BTS
Video editing
Synchronize visual track with musical beats
Useful for creating MTV-style video

Audio editing
Handle acoustic signals as a set of beats

Stage lighting control

6. Applications of BTS

No.17

Real-time computer graphics
Create real-time CG synchronized with music
Display a virtual dancer
whose motion is changed with beats in real time
BDMusical Acoustic Signals

SDBDSDBDSDBDSD

Beat Information
BTSCD

AP1000

Animation ServerFujitsu AP1000
SGI Indigo2

No.18

7. Conclusion
Track beats in acoustic signals
Manage multiple beat-tracking agents
Automatically learn characteristics of BD and SD
Useful for various multimedia applications

7. Conclusion

No.19

Future Work
Reduce limitations on the input
Recognize higher level musical events
Apply to multimedia systems7. Conclusion

No.20