IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

333

Drum Sound Recognition for Polyphonic Audio
Signals by Adaptation and Matching of Spectrogram
Templates With Harmonic Structure Suppression
Kazuyoshi Yoshii, Student Member, IEEE, Masataka Goto, and Hiroshi G. Okuno, Senior Member, IEEE
Abstract—This paper describes a system that detects onsets ofthe bass drum, snare drum, and hi-hat cymbals in polyphonicaudio signals of popular songs. Our system is based on a template-matching method that uses power spectrograms of drumsounds as templates. This method calculates the distance between
a template and each spectrogram segment extracted from a song
spectrogram, using Goto’s distance measure originally designed
to detect the onsets in drums-only signals. However, there are twomain problems. The first problem is that appropriate templates
are unknown for each song. The second problem is that it is moredifficult to detect drum-sound onsets in sound mixtures including
various sounds other than drum sounds. To solve these problems,
we propose template-adaptation and harmonic-structure-suppression methods. First of all, an initial template of each drum sound,called a seed template, is prepared. The former method adapts it
to actual drum-sound spectrograms appearing in the song spectrogram. To make our system robust to the overlapping of harmonicsounds with drum sounds, the latter method suppresses harmoniccomponents in the song spectrogram before the adaptation andmatching. Experimental results with 70 popular songs showedthat our template-adaptation and harmonic-structure-suppression
methods improved the recognition accuracy and achieved 83%,58%, and 46% in detecting onsets of the bass drum, snare drum,
and hi-hat cymbals, respectively.
Index Terms—Drum sound recognition, harmonic structure suppression, polyphonic audio signal, spectrogram template, template
adaptation, template matching.

I. INTRODUCTION
HE importance of music content analysis for musicalaudio signals has been increasing in the field of music
information retrieval (MIR). MIR aims at retrieving musicalpieces by executing a query about not only text informationsuch as artist names and music titles but also musical contentssuch as rhythms and melodies. Although the amount of digitally
recorded music available over the Internet is rapidly increasing,
there are only a few ways of using text information to efficiently

T

Manuscript received February 1, 2005; revised December 19, 2005. This
work was supported in part by the Ministry of Education, Culture, Sports,
Science and Technology (MEXT), Grant-in-Aid for Scientific Research (A)15200015 and by the COE Program of MEXT, Japan. The associate editor
coordinating the review of this manuscript and approving it for publication was
Dr. Michael Davies.
K. Yoshii and H. G. Okuno are with the Department of Intelligence Scienceand Technology, Graduate School of Informatics, Kyoto University, Kyoto 6068501, Japan (e-mail: yoshii@kuis.kyoto-u.ac.jp; okuno@i.kyoto-u.ac.jp).
M. Goto is with the National Institute of Advanced Industrial Science andTechnology (AIST), Tsukuba 305-8568, Japan (e-mail: m.goto@aist.go.jp).
Digital Object Identifier 10.1109/TASL.2006.876754

find our desired musical pieces in a huge music database. Music
content analysis enables MIR systems to automatically understand the contents of musical pieces and to deal with them even
if they do not have metadata about the artists and titles.
As the first step of achieving content-based MIR systems in
the future, we focus on detecting onset times of individual musical instruments. In this paper, we call this process recognition, which means simultaneous processing of both onset detection and identification of each sound. Although onset time information of each musical instrument is low-level musical content, the recognition results can be used as a basis for higherlevel music content analysis concerning the rhythm, melody,and chord, such as beat tracking, melody detection, and chord
change detection.
In this paper, we propose a system of recognizing drumsounds in polyphonic audio signals sampled from commercial
compact-disc (CD) recordings of popular music. We allow
various music styles for popular music, such as rock, dance,
house, hip-hop, eurobeat, soul, R&B, and folk. Our system
detects onset times of three drum instruments—bass drum,
snare drum, and hi-hat cymbals—while identifying them. Fora large class of popular music with drum sounds, these three
instruments play important roles as the rhythmic backboneof music. We believe that accurate onset detection of drumsounds is useful for describing temporal musical contents suchas rhythm, tempo, beat, and measure. Previous studies [1]–[4]on describing those temporal contents, however, have focusedon the periodicity of time-frame-based acoustic features, andhave not tried to detect accurate onset times of drum sounds.
Previous studies [5], [6] on genre classification did not consider
onset times of drum sounds while such onset times could be
used for improving classification performances by identifying
drum patterns unique to musical genres. Some recent studies[7], [8] reported the use of drum patterns for genre classification
while Ellis et al. [7] dealt with only MIDI signals. The resultsof our system are useful for such genre classification withhigher-level content analysis of real-world audio signals.
The rest of this paper is organized as follows. In Section II,
we describe the current state of drum sound recognition techniques. In Section III, we examine the problems and solutions ofrecognizing drum sounds contained in commercial CD recordings. Sections IV and V describe the proposed solutions: template-adaptation and template-matching methods, respectively.
Section VI describes a harmonic-structure-suppression method
to improve the performance of our system. Section VII shows
experimental results of evaluating these methods. Finally, Section VIII summarizes this paper.

1558-7916/$20.00 © 2006 IEEE

334

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

II. ART OF DRUM SOUND RECOGNITION
We start on describing the current state of the art of drumsound recognition and related work motivating our approach.
A. Current State
Although there are many studies on onset detection or identification of drum sounds, a few of them have dealt with drumsound recognition for polyphonic audio signals such as commercial CD recordings. The drum sound recognition method byGoto and Muraoka [9] was the earliest work that could deal with
drum-sound mixtures of solo performances with MIDI rockdrums. Herrera et al. [10] compared conventional feature-basedclassifiers in the experiments of identifying monophonic drumsounds. To recognize drum sounds in drums-only audio signals, various modeling methods such as N-grams [11], probabilistic models [12], and SVM [13] have been used. By usinga noise-space-projection method, Gillet and Richard [14] tried
to recognize drum sounds in polyphonic audio signals. Thesestudies, however, cannot fully deal with both the variation ofdrum-sound features and their distortion caused by the overlapping of other sounds.
The detection of bass and snare drum sounds in polyphonic
CD recordings was mentioned in Goto’s study on beat tracking[15]. Since it roughly detected them to estimate a hierarchicalbeat structure, the accurate drum detection was not investigated. Gouyon et al. [16] proposed a method that classifies
mixed sounds extracted from polyphonic audio signals into two
categories of the bass and snare drums. As the former step of
the classification, they proposed a percussive onset detection
method. It was based on a unique idea of template adaptationthat can deal with drum-sound variations according to musicalpieces. Zils et al. [17] tried the extraction and resynthesis ofdrum tracks from commercial CD recordings by extending
Gouyon’s method, and showed the promising results.
To recognize drum sounds in audio signals of drum tracks,
sound source separation methods have been focused. They madevarious assumptions in decomposing a single music spectrogram into multiple spectrograms of musical instruments; independent subspace analysis (ISA) [18], [19] assumes the statistical independence of sources, non-negative matrix factorization (NMF) [20] assumes their non-negativity, and sparse
coding combined with NMF [21] assumes their non-negativity
and sparseness. Further developments were made by FitzGerald
et al. [22], [23]. They proposed PSA (Prior Subspace Analysis) [22] that assumes prior frequency characteristics of drumsounds, and applied it to recognize drum sounds in the presenceof harmonic sounds [23]. For the same purpose, Dittmar andUhle [24] adopted non-negative independent component analysis (ICA) that considers the non-negativity of sources. In thesestudies, the recognition results depend not only on the separation quality but also on the reliability of estimating the numberof sources and classifying them. However, the estimation andclassification methods are not robust enough for the sake ofrecognizing drum sounds in audio signals containing time-frequency-varying various sounds.
Klapuri [25] reported a method of detecting onsets of allsounds in polyphonic audio signals. Herrera et al. [26] used

Klapuri’s algorithm to estimate the amount of percussive onsets. However, drum sound identification was not evaluated. To
identify drum sounds extracted from polyphonic audio signals,
Sandvold et al. [27] proposed a method that adapts featuremodels to those of drum sounds used in each musical piece, butthey used correct instrument labels for the adaptation.
B. Related WorkWe explain two related methods in detail.
1) Drum Sound Recognition for Solo Drum Performances:
Goto and Muraoka [9] reported a template-matching method forrecognizing drum sounds contained in musical audio signals of
popular-music solo drum performances by a MIDI tone generator. Their method was designed in the time-frequency domain.
First, a fixed-time-length power spectrogram of each drum to berecognized is prepared as a spectrogram template. There werenine templates corresponding to nine drum instruments (bassand snare drums, toms, and cymbals) in a drum set. Next, onset
times are detected by comparing the template with the power
spectrogram of the input audio signal, assuming that the input
signal is a polyphonic sound mixture of those templates. In thetemplate-matching stage, they proposed a distance measure (wecall this “Goto’s distance measure” in this paper), which is robust for the spectral overlapping of a drum sound corresponding
to the target template with other drum sounds.
Although their method achieved the high recognition accuracy, it has a limitation that the power spectrogram of each drum
used in the input audio signal must be registered with the recognition system. In addition, it has difficulty recognizing drumsounds included in polyphonic music because it does not assume the spectral overlapping of harmonic sounds.
2) Drum Sound Resynthesis From CD Recordings: Zils et al.
[17] reported a template-adaptation method for recognizing bass
and snare drum sounds from polyphonic audio signals sampledfrom popular-music CD recordings. Their method is defined in
the time domain. First, a fixed-time-length signal of each drumis prepared as a waveform template, which is different from an
actual drum signal used in a target musical piece. Next, by calculating the correlation between each template and the musicalaudio signal, onset times at which the correlation is large are detected. Finally, a drum sound is created (i.e., the signal templateis updated) by averaging fixed-time-length signals starting fromthose detected onset times. These operations are repeated until
the template converges.
Although their time-domain analysis seems to be promising,it has limitations in dealing with overlapping drum sounds in thepresence of other musical instrument sounds.
III. DRUM SOUND RECOGNITION PROBLEM
FOR POLYPHONIC AUDIO SIGNALSFirst, we define the task of our drum sound recognition
system. Next, we describe the problems and solutions in recognizing drum sounds in polyphonic audio signals.
A. Target
The purpose of our research is to detect onset times of threekinds of drum instruments in a drum set: bass drum, snare drum,
and hi-hat cymbals. Our system takes polyphonic musical audio

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

Fig. 1. Overview of drum sound recognition system: a drum-sound spectrogram
template (input) is adapted to actual drum-sound spectrograms appearing in thesong spectrogram (input) in which the harmonic structure is suppressed. The
adapted template is compared with the song spectrogram to detect onsets (output).

signals as input, which are sampled from popular-music CD
recordings and contain sounds of vocal parts and various musical instruments (e.g., piano, trumpet, and guitar) as well as
drum sounds. Drum sounds are performed by real drum sets
(e.g., popular/rock drums) or electronic instruments (e.g., MIDI
tone generators). Assuming the main target is popular rock-style
music, we focus on the basic playing style of drum performances
using normal sticks, and do not deal with special playing styles(e.g., head-mute and brush).
B. Problems
In this paper, we develop a template-based recognition systemthat defines a template as a fixed-time-length power spectrogramof each drum: bass drum, snare drum, or hi-hat cymbals. There
are the following two problems, considering the discussion in
Section II-B.
1) Individual Difference Problem: Acoustic features of drumsounds vary among musical pieces and the appropriate templates for recognizing drum sounds in each piece are usually
unknown in advance.
2) Mixed Sound Problem: It is difficult to accurately detect drum sounds included in polyphonic audio signals because
acoustic features are distorted by the overlapping of other musical instrument sounds.
C. Approach
We propose an advanced template-adaptation method to solve
the individual difference problem described in Section III-B.
After performing the template adaptation, we detect onset timesof drum sounds using an advanced template-matching method. In
addition, in order to solve the mixed sound problem, we proposea harmonic-structure-suppression method that improves therobustness of our adaptation and matching methods. Fig. 1 shows
an overview of our proposed drum sound recognition system.
1) Template Adaptation: The purpose of this adaptation is
to obtain a spectrogram template that is adapted to its corresponding drum sound used in the polyphonic audio signal ofa target musical piece. Before the adaptation, we prepare individual spectral templates (we call seed-templates) for bass
drum, snare drum, and hi-hat cymbals; three templates in total.
To adapt the seed-templates to the actual drum sounds, we extended Zils’ method to the time-frequency domain.
2) Template Matching: The purpose is to detect all the onset
times of drum sounds in the polyphonic audio signal of the335

target piece, even if other musical instrument sounds overlap the
drum sounds. By using Goto’s distance measure considering thespectral overlapping, we compare the adapted template with thespectrogram of the audio signal. We present an improved spectral weighting algorithm based on Goto’s algorithm for use incalculating the matching distance.
3) Harmonic Structure Suppression: The purpose is to suppress harmonic components of other instrument sounds in theaudio signal when recognizing sounds of bass and snare drums.
In the recognition of hi-hat cymbal sounds, this processing is
not performed under the assumption that harmonic components
are weak enough at a high-frequency band.
We use two different distance measures between the template
adaptation and matching stages. In the adaptation stage, it isdesirable to detect only semi-pure drum sounds that have little
overlap with other sounds. Those drum sounds tend to resultin a good adapted template that includes little spectral components of other sounds. Because it is not necessary to detect all
the onset times of a target drum instrument, a distance measureused in this stage does not care about the spectral overlappingof other sounds. In the matching stage, on the other hand, weused the Goto’s distance measure because it is necessary to exhaustively detect all the onset times even if target drum sounds
are overlapped by other sounds.
The recognition of bass drum, snare drum, and hi-hat cymbal
sounds is performed separately. In the following sections, theterm “drum” means one of these three drum instruments.
IV. TEMPLATE ADAPTATIONA drum sound template is a power spectrogram in thetime-frequency domain. Our template-adaptation method uses
a single initial template, called a “seed template,” for each kindof drum instruments. To recognize the sounds of the bass drum,
snare drum and hi-hat cymbals, for example, we require justthree seed templates, each of which is individually adapted byusing the method.
Our method is based on an iterative adaptation algorithm.
An overview of the method is shown in Fig. 2. First, OnsetCandidate-Detection stage roughly detects onset candidates in
the input audio signal of a musical piece. Starting from each
onset candidate, a spectrogram segment whose time-length isfixed is extracted from the power spectrogram of the input audio
signal. Then, by using the seed template and all the spectrogram segments, the iterative algorithm successively applies twostages—Segment Selection and Template Updating—to obtain
the adapted template.
1) The Segment-Selection stage estimates the reliability thateach spectrogram segment includes the drum sound spectrogram. The spectrogram segments with high reliabilities
are then selected: this selection is based on a fixed ratio tothe number of all the spectrogram segments.
2) The Template-Updating stage then reconstructs an updated
template by estimating the power that is defined, at eachframe and each frequency, as the median power among
the selected spectrogram segments. The template is thus
adapted to the current piece and used for the next adaptive
iteration.

336

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

Fig. 3. Lowpass filter functions F , F
which represent typical frequency
characteristics of bass and snare drum sounds, and highpass filter function F
which represents that of hi-hat cymbal sounds.

Fig. 2. Overview of template-adaptation method: each template is represented
as a fixed-time-length power spectrogram in the time-frequency domain. Thismethod adapts a single seed template corresponding to each drum instrument toactual drum sounds appearing in a target musical piece. The method is based on
an iterative adaptation algorithm, which successively applies two stages—Segment Selection and Template Updating—to obtain the adapted template.

is a lowpass or highpass
where
filter function, as is shown in Fig. 3. We assume that it represents the typical frequency characteristics of bass drumsounds (BD), snare drum sounds (SD) or hi-hat cymbal
sounds (HH).
3) Each onset time is given by the time found by peak-picking.
is smoothed by Savitzky and Golay’sinsmoothing method [28] before its peak time is calculated.
B. Preparing Seed Templates and Spectrogram Segments

A. Onset Candidate Detection
To reduce the computational cost of the template matching,
the Onset-Candidate-Detection stage detects possible onset
times of drum sounds as candidates: the template matchingis performed only at these onset candidates. For the purposeof detecting onset times, Klapuri’s method [25] is often used,
but we adopted an easy peak-picking method [9] to detect
onset candidate times. The reason is that it is important tominimize the detection failure (miss) of actual drum-soundonsets; the high recall rate is preferred even if there are many
false alarms. Note that each detected onset candidate does
not necessarily correspond to an actual drum-sound onset.
The template-matching method judges whether each onset
candidate is an actual drum-sound onset.
The time at which the power takes a local maximum value isdenote the power at
detected as an onset candidate. Let
be its time differenframe and frequency bin , andtial. At every frame (441 points),
is calculated by applying the short-time Fourier transformation (STFT) with Hanning windows (4096 points) to the signal sampled at 44.1 kHz.
In this paper, we use log scale [dB] as the power unit. The onset
candidate times are then detected as follows:
is satisfied for three consecutive frames
1) If,
is defined as(the sub1) Seed Template Construction: Seed template
script means seed) is an average power spectrogram preparedfor each drum type to be recognized. The time-length (frames)
is fixed.
is represented as a time-freof seed template(quency matrix whose element is denoted asframes ,bins ).
To create seed template , it is necessary to prepare multiple
drum sounds each of which contains a solo tone of the drumsound. We used drum-sound samples taken from “RWC Music
Database: Musical Instrument Sound” (RWC-MDB-I-2001).
They were performed in a normal style on six different real
drum sets. By applying the onset candidate detection method,
an onset time in each sample is detected. Starting from eachtime, a power spectrogram whose size is the same as the seed
template, is calculated by executing STFT. Therefore, multiple
power spectrograms of monophonic drum sounds are obtained,, whereeach of which is denoted asmeans the number of the extracted power spectrograms (thenumber of the prepared drum sounds).
Because there are timbre variations of drum sounds, we used
multiple drum-sound spectrograms in constructing seed temis calculated
plate . Therefore, in this paper, seed templateby collecting the maximum power of the power spectrograms
at each frame and each frequency bin

(1)

(3)

Otherwise,.
2) At every frame , the weighted summation
is calculated by

In the iterative adaptation algorithm, let denote a templateis the first tembeing adapted after th iteration. Because
is set to. We also obtain power spectrogram
plate,
weighted by filter function

of

(2)
(4)

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

3372) Spectrogram Segment Extraction: The th spectrogramis a power spectrogram via STFT
segment
starting from an onset candidate time [ms] in the audio signalof a target musical piece, whereis the number of the onset
candidates. The size of each spectrogram segment is the samewith that of seed template , and thus it is also represented asa time-frequency matrix. We also obtain power spectrogram
weighted by filter function(5)
C. Segment Selectionthat spectrogram segment
includes the
The reliability
spectral components of the target drum sound is estimated, andthen spectrogram segments are selected in descending order. The ratiowith respect to the reliabilitiesof the number of the selected segments to the number of all the)
spectrogram segments (the number of the onset candidates:
is fixed. In this paper, the ratio is empirically set to 0.1 (i.e., the).
number of the selected segments is
as the reciprocal of the distanceWe define the reliability
between templateand spectrogram segment(6)
is reThe distance measure used in calculating the distancequired to satisfy that, if the reliability that spectrogram segment
includes the drum sound spectrogram becomes large, the disbecomes small. We describe the individual distancetance
measurement for each drum sound recognition.
1) In Recognition of Bass and Snare Drum Sounds: In thefirst adaptive iteration, typical spectral distance measures (e.g.,
Euclidean distance measure) cannot be applied to calculate thebecause those measures inappropriately make thedistancelarge even if spectrogram segment
includes the
distance
target drum sound spectrogram. In general, the power spectrogram of bass or snare drum sounds has salient spectral peaks
that depend on the kind of drum instrument. Because seed temhas never been adapted, the spectral peak positions ofplate
are different from those of the target drum sound spectrolarge. On the other hand,gram, which makes the distanceif spectral peaks of other musical instruments in a spectrogram
segment happen to overlap the salient peaks of seed template, the distancebecomes small, which results in selecting
inappropriate spectrogram segments.
To solve this problem, we perform spectral smoothing in aand eachlower time-frequency resolution for seed template
spectrogram segment . In this paper, the time resolution is 2
[frames] and the frequency resolution is 5 [bins] in the spectral
smoothing, shown in Fig. 4. This processing allows for differences in the spectral peak positions between seed templateand each spectrogram segment
and inhibits the undesirable
when a spectrogram segment inincrease of the distancecludes the drum sound spectrogram.
anddenote the smoothed seed template and aLetsmoothed spectrogram segment.
in a time-frequency

Fig. 4. Spectral smoothing at a lower time-frequency resolution in the Segment-Selection stage in bass and snare drum sound recognition: this inhibits theundesirable increase of distance between seed template and spectrogram segment which includes a drum sound spectrogram.

range

,
is calculated by(7)
is calculated in the same way. This operation means
the averaging and reallocation of the power, shown in Fig. 4.
First, the time-frequency domain is separated into rectangularsectors. The size of each sector is 2 [frames] 5 [bins]. Next, theaverage power in each sector is calculated, and then reallocatedto each bin in that sector.
between seed templateandThe spectral distancespectrogram segment in the first iteration is defined as(8)
After the first iteration, we can use the Euclidean distancemeasure without the spectral smoothing because the spectral
peak positions of template
are adapted to those of
the drum sound used in the audio signal. The spectral distance
between templateand spectrogram segmentin theth adaptive iteration is defined as(9)
To focus on the precise characteristic peak positions of the drumsound used in the musical performance, we do not use the spectral smoothing in the equation (9). Because those positions areuseful for selecting appropriate spectrogram segments, it is desirable that the equation (9) reflects the differences of the spectral peak positions between the template and a spectrogram segment to the distance.
2) In Recognition of Hi-Hat Cymbal Sounds: The spectral
in any adaptive iteration is always calculated after
distanceand spectrogram segthe spectral smoothing for template
ment . In this paper, the time resolution is 2 [frames] and thefrequency resolution is 20 [bins] in the spectral smoothing. A

338

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

smoothed template
and a smoothed spectrogram segment
are obtained in the similar way of smoothing thespectrogram of bass and snare drum sounds. Using these specbetween templateandtrograms, the spectral distanceis defined as
spectrogram segment(10)

In general, the power spectrogram of hi-hat cymbal sounds
seems not to have salient spectral peaks such as those of bass andsnare drum sounds. We think it is more appropriate to focus on
the shape of the spectral envelope than the fine spectral structure.
To ignore the large variation of the local spectral component ina small time-frequency range and extract the spectral envelope,the spectral smoothing is necessary.

Fig. 5. Updating template by collecting the median power at each frame andeach frequency bin among selected spectrogram segments: harmonic components are suppressed in the updated template.

D. Template Updating
An updated template is constructed by collecting the median
power at each frame and each frequency bin among all the selected spectrogram segments. The updated template is used as
the template in the next adaptive iteration. We describe updating
algorithms for the template of each drum sound.
1) In Recognition of Bass and Snare Drum Sounds: The upwhich is weighted by filter functionisdated templateobtained by(11)
are the spectrogram segments sewhereis the number of thelected in the Segment-Selection stage.
in this paper.
selected spectrogram segments, which is
We pick out the median power at each frame and each frequency bin because we can suppress spectral components that
do not belong to the target drum sound spectrogram (Fig. 5).
A spectral structure of the target drum sound spectrogram (e.g.,salient spectral peaks) can be expected to appear as the same
spectral shape in most selected spectrogram segments. On theother hand, spectral components of other musical instrument
sounds appear at different frequencies among spectrogram segments. In other words, the local power at the same frame andthe same frequency in many spectrogram segments is exposedas the power of the pure drum sound spectrogram. By picking
out the median of the local power, unnecessary spectral components of other musical instrument sounds become outliers andare not picked out. We can thus obtain a template which is close
to the solo drum sound spectrogram even if various instrument
sounds are included in the musical audio signal.
2) In Recognition of Hi-Hat Cymbal Sounds: The updated
that is weighted by filter function
and smoothed templateis obtained by(12)

Fig. 6. Overview of template-matching method: each spectrogram segment iscompared with the adapted template by using Goto’s distance measure to detect
actual onset times. This distance measure can appropriately determine whether
the adapted template is included in a spectrogram segment even if there are othersimultaneous sounds.

If spectrogram segments are not smoothed, the stable median power cannot be obtained because the local power in the
spectrogram of hi-hat cymbal sounds varies among onsets. By
smoothing the spectrogram segments, the median power is determined as a stable value because the shape of the spectral envelope obtained by the spectral smoothing is stable in the spectrogram of hi-hat cymbal sounds.
V. TEMPLATE MATCHING
To find actual onset times, this method judges whether the
drum sound actually occurs at each onset candidate time,shown in Fig. 6. This alternative determination is difficult
because other various sounds often overlap the drum sounds.
If we use a general distance measure, the distance between
the adapted template and a spectrogram segment including thetarget drum sound spectrogram becomes large when there aremany other sounds that are simultaneously performed withthe drum sound. In other words, the overlapping of the other
instrument sounds makes the distance large even if the target
drum sound spectrogram is included in a spectrogram segment.

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

339

Fig. 8. Examples of adapted templates of bass drum (left), snare drum
(center) and hi-hat cymbals (right): these spectrograms show that characteristicfrequency bins are different among three drum instruments.

B. Power Adjustment of Spectrogram Segments
Fig. 7. Power adjustment of spectrogram segments: if a spectrogram segment
includes the drum sound spectrogram, the power adjustment value is large (top).
Otherwise, the power adjustment value is small (bottom).

To solve this problem, we adopt a distance measure proposedby Goto et al. [9]. Because Goto’s distance measure focuseson whether the adapted template is included in a spectrogram
segment, it can calculate an appropriate distance even if thedrum sound is overlapped by other musical instrument sounds.
We present an improved method for selecting characteristic frequencies. In addition, we propose a thresholding method that
automatically determines appropriate thresholds for each musical piece.
An overview of our method is shown in Fig. 6. First, WeightFunction-Preparation stage generates a weight function which
represents spectral saliency of each spectral component in theadapted template. This function is used for selecting characteristic frequency bins in the template. Next, Power-Adjustment
stage calculates the power difference between the template andeach spectrogram segment by focusing on the local power difference at each characteristic frequency bin (Fig. 7). If the power
difference is larger than a threshold, it judges that the drumsound spectrogram does not appear in that segment, and does
not execute the subsequent processing. Otherwise, the power ofthat segment is adjusted to compensate for the power difference.
Finally, Distance-Calculation stage calculates the distance between the adapted template and each adjusted spectrogram segment. If the distance is smaller than a threshold, it judges thatthe drum sound spectrogram is included.
In this section, we describe a template-matching algorithm
for bass and snare drum sound recognition. In hi-hat cymbal
sound recognition, the adapted template is obtained as thesmoothed spectrogram. Therefore, a template-matching algorithm for hi-hat cymbal sound recognition is obtained by,).
replacing with in each expression (e.g.,
A. Weight Function Preparation
A weight function represents the spectral saliency at eachframe and frequency bin in the adapted template. The weightfunction is defined as

The power of each spectrogram segment is adjusted to matchwith that of the adapted template by assuming that the drumsound spectrogram is included in that spectrogram segment.
This adjustment is necessary to correctly determine that theadapted template is included in a spectrogram segment even if
the power of the drum sound spectrogram included in that spectrogram segment is smaller than that of the template. On theother hand, if the drum sound spectrogram is not actually included in a spectrogram segment, the power difference is expected to be large. Therefore, if the power difference is larger
than a threshold, we determine that the drum sound spectrogram
is not included in that spectrogram segment.
To calculate the power difference between each spectrogram
and template, we focus on the local power difsegmentin theferences at spectral characteristic frequency bins oftime-frequency domain. The algorithm of the power adjustment
is described as follows:
1) Selecting Characteristic Frequency Bins in Adapted Tembe the characteristic frequency
plate: Letbins in the adapted template, whereis the number of characteristic frequency bins at each frame.
,,. Fig. 8
In this paper,shows the differences of characteristic frequency bins amongis determined at each frame .
three drum instruments.
is selected as a frequency bin whereis the th largest
which satisfies the following conditions:
among
(14)
(15)
(16)
where
is a constant, which is set to 0.5 in this paper. Theseshouldthree conditions (14), (15), and (16) mean thatbe peaked along the frequency direction.
2) Calculating Power Difference: The local power difference
at frame and characteristic frequency bin
is calculated as(17)
The local-time power difference
as the first quartile ofat frame is determined

first-quartile(18)

arg-first-quartile

(19)

(13)
where
represents the adapted template which is weighted byfilter function.

340

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

where
is whenis the first quartile. If theis satisfied is larger thannumber of frames wherea threshold , we determine that the template is not included
is a threshold autoin that spectrogram segment, whereis set to 5 [frames]
matically determined in Section V-D andin this paper.
We pick out not the minimum but the first quartile among
because thethe power differences
latter value is more robust for outliers included in them. The
power difference at a characteristic frequency bin may becomelarge when harmonic components of other musical instrument
sounds accidentally exist at that frequency. Picking out the firstquartile ignores the accidental large power difference and extracts the essential power difference derived from whether thetemplate is included in a spectrogram segment or not.
3) Adjusting Power of Spectrogram Segments: The total
power differenceis calculated by integrating the local-time
which satisfies, weighted
power difference
by weight function(20)
is satisfied, we are able to determine that the temIfisplate is not included in that spectrogram segment, wherea threshold automatically determined in Section V-D.
denote an adjusted spectrogram segment after theLetpower adjustment, obtained byis calculated by integrating the local
The total distancedistance in the time-frequency domain, weighted by weightfunction(23)
To determine whether the targeted drum sound occurred at atime corresponding to the spectrogram segment , the distanceis compared with a threshold. If
is satisfied,
we conclude that the targeted drum sound occurred.
is alsoautomatically determined in Section V-D.
D. Automatic Thresholding,
To determine 12 thresholds (and) that are optimized for each musical piece, we use athreshold selection method proposed by Otsu [29]. It is better todynamically change the thresholds to yield the best recognition
results for each piece.
By using Otsu’s method, we determine each optimized,or) which classifies a set ofthreshold (,orvalues () into two classes: the one class contains
values which are less than the threshold, the other contains
the rest of values. We define a threshold which maximizes
the between-class variance (i.e., minimizes the within-classvariance).
Finally, to balance the recall rate with the precision rate (theseandrates are defined in Section VII-A), we adjust thresholds
which are determined by Otsu’s method

(21)
(24)

C. Distance Calculation
To calculate the distance between adapted templateandan adjusted spectrogram segment , we adopt Goto’s distancemeasure [9]. It is useful for judging whether the adapted template is included in each spectrogram segment or not (the answer
is “yes” or “no”). Goto’s distance measure does not make thedistance large even if the spectral components of the target drumissound are overlapped with those of other sounds. If, Goto’s distance measure regards
larger thanas a mixture of spectral components not only of the drum soundbut also of other musical instrument sounds. In other words,
includes, then the local
when we identify that
distance at frame and frequency bin is minimized. Therefore, the local distance measure is defined as(22)otherwise
whereis the local distance at frame and frequency bin. The negative constantmakes thisdistance measure robust for the small variation of local spectral
is larger than about,
components. If
becomes zero. In this paper,
dB ,
dB .

and
are empirically determined scaling (balwhereancing) factors, which are described in Section VII-B.
VI. HARMONIC STRUCTURE SUPPRESSION
Our proposed method of suppressing harmonic components improves the robustness of the template-adaptation andtemplate-matching methods for the spectral overlapping of harmonic instrument sounds. Real-world CD recordings usually
include many harmonic instrument sounds. If the combined
power of various harmonic components is much larger than thatof the drum sound spectrogram in a spectrogram segment, it isoften difficult to correctly detect the drum sound. Therefore, therecognition accuracy is expected to be improved by suppressing
those unnecessary harmonic components.
To suppress harmonic components in a musical audio
signal, we sequentially perform three operations for each
spectrogram segment: estimating F0 of harmonic structure,verifying harmonic components, and suppressing harmoniccomponents. These operations are enabled in bass and snare
drum sound recognition. In hi-hat cymbal sound recognition,
the harmonic-structure-suppression method is not necessary
because most influential harmonic components are expected to.
be suppressed by the highpass filter function

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

341

A. F0 Estimation of Harmonic Structure
The F0 is estimated at each frame by using a comb-filter-like
spectral analysis [30], which is effective in roughly estimating
predominant harmonic structures in polyphonic audio signals.
that theThe basic idea is to evaluate the reliability
frequency is the F0 at each frame and each frequency .
is defined as the summation of the
The reliability
local amplitude weighted by a comb-filter

Fig. 9. Suppressing hth harmonic component of the F0 F (t) by linearlyinterpolating between the minimum power on both sides of spectral peak.

(25)
is [cent],1 and each inwhere the frequency unit of andis thecrement of is 100 [cent] in the summation.
local amplitude at frame and frequency [cent] in a spectrodenotes a comb-filter-like function
gram segment .
which passes only harmonic components which form the harmonic structure of the F0frame in the neighborhood of a th harmonic component of thecent tocent in our implementation)
F0 (fromis calculated. Second, we determine that the th harmonic component of the F0 at frame is actually derived from only haris larger than a threshold,monic instrument sounds if
which is set to 2.0 in this paper (c.f., the kurtosis of the Gaussian
distribution is 3.0).
C. Harmonic Component Suppression

(26)
(27)
where is the number of harmonic components considered andis an amplitude attenuation factor. The spectral spreading of.
is aeach harmonic component is represented by
Gaussian distribution, where is the mean and is the standard,,
cent .
deviation. In this paper,
Frequenciesof the F0 are determined by finding frequencies that satisfy the following condition:
(28)
where
is a constant, which is set to 0.7 in this paper. The F0
is searched from 2000 [cent] (51.9 [Hz]) to 7000 [cent] (932 Hz)
by shifting every 100 [cent].
B. Harmonic Component Verification
It is necessary to verify that each harmonic component estimated in Section VI-A is actually derived from only harmonic
instrument sounds. To suppress all the estimated harmonic components without this verification is not appropriate because acharacteristic frequency of drum sounds may be erroneously estimated as a harmonic frequency if the power of drum sounds
is much larger than that of harmonic instrument sounds. In another case, a characteristic frequency of drum sounds may be
accidentally equal to a harmonic frequency. The verification ofeach harmonic component prevents characteristic spectral components of drum sounds from being suppressed.
We focus on the general fact that spectral peaks of harmoniccomponents are much more peaked than characteristic spectralatpeaks of drum sounds. First, the spectral kurtosis
1Frequency f

1200 log (fin hertz is converted to frequency fcent in cents: fcent =)).

=(440 2 2

We suppress harmonic components that are identified
as being actually derived from only harmonic instrument
sounds. An overview is shown in Fig. 9. First, we find thetwo frequencies of the local minimum power adjacent to thespectral peak corresponding to each harmonic component at
cent . Second, we linearly interpolate thepower between them along the frequency axis while preserving
the original phase.
VII. EXPERIMENTS AND RESULTS
We performed experiments of recognizing the bass drums,
snare drums, and hi-hat cymbals for polyphonic audio signals.
A. Experimental Conditions
We tested our methods on seventy songs sampled fromthe popular music database “RWC Music Database: PopularMusic” (RWC-MDB-P-2001) developed by Goto et al. [31].
Those songs contain sounds of vocals and various instruments
as songs in commercial CDs do. Seed templates were createdfrom solo tones included in “RWC Music Database: MusicalInstrument Sound” (RWC-MDB-I-2001) [32]: a seed templateof each drum is created from multiple sound files each ofwhich contains a sole tone of the drum sound by normal-styleperformance. All original data were sampled at 44.1 kHz with
16 bits, stereo. We converted them to monaural recordings.
We evaluated the experimental results by the recall rate, precision rate and f-measurerecall rate
rate
f-measurecorrectly detected onsets
actual onsets
correctly detected onsets
detected onsets
recall rate precision rate
recall rate precision rate

To prepare actual onset times (correct answers), we extracted
onset times (note-on events) of the bass drums, snare drums,

342IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

TABLE INUMBER OF ACTUAL ONSETS IN 70 MUSICAL PIECESTABLE IISETTING OF COMPARATIVE EXPERIMENTS

and hi-hat cymbals from the standard MIDI files of the seventy songs, which are distributed with the music database, andaligned them to the corresponding audio signals by hand. The
number of actual onsets of each drum sound included in seventy
songs is shown in Table I (about 100 000 onsets in total). If thedifference between a detected onset time and an actual onset
time was less than 25 [ms], we judged that the detected onset
time is correct.
B. Experimental Results
To evaluate our proposed three methods: template-matching
method ( -method), template-adaptation method ( -method),and harmonic-structure-suppression method ( -method), we
performed comparative experiments by enabling each method
one by one: we tested three procedures shown in Table II,-procedure,-procedure, and-procedure. The-procedure was not tested for recognizing hi-hat cymbal
sounds because the -method is enabled only for recognizing
bass or snare drum sounds. The -procedure used a seed template instead of the adapted template for the template-matching.
andwere determined for each
The balancing factors
experiment as shown in Table III.
For convenience, we evaluated three procedures by dividing
70 musical pieces into three groups: group I, II, and III. First,
70 pieces were sorted in descending order with respect to the-procedure
f-measure by the fully-enabled procedure (i.e.,-procedure inin bass and snare drum sound recognition,
hi-hat cymbal sound recognition). Second, the first 20 pieces
were put in group I, and the next 25 pieces were put in group II,
and the remaining 25 pieces were put in group III.
The average recall and precision rates of onset candidate detection was 88%/22% (bass drum sound recognition), 77%/18%(snare drum sound recognition), and 87%/36% (hi-hat cymbal
sound recognition). This means the chance rates of onset detection by the coin-toss decision were 29%, 25%, and 39%, respectively. Table III shows the experimental results obtained byeach procedure. Table IV shows the recognition error reduction rates which represent the f-measure improvement obtainedby enabling the -method added to the -procedure, and that-proceobtained by enabling the -method added to thedure. Table V shows a complete list of musical pieces sorted
in descending order with respect to f-measure of each drum instrument recognition. Fig. 10 shows f-measure curves along thesorted musical pieces in recognizing each drum instrument.

C. Discussion
The experimental results show the effectiveness of our-procedures
methods. In general, the fully-enabledyielded the best performance in bass and snare drum soundrecognition. In these case, the average f-measure was 82.924%
and 58.288%, respectively. In hi-hat cymbal sound recognition-procedure, the average f-measure was 46.249%. In
by thetotal, the f-measure averaged over those three drum instruments
was about 62%. In our observation, the effectiveness of the
A-method and S-method was almost independent to specificplaying styles. If harmonic sounds which mainly distribute ina low frequency band (e.g., spectral components of bass line)
are more dominant, the suppression method tends to be more
effective. We discuss in detail in the following sections.
1) Bass Drum Sound Recognition: The f-measure in bass
drum sound recognition (82.92% in total) was highest among
the results of recognizing three drum instruments. Table IV
showed that both the -method and the -method were very effective, especially in group I. It also showed that the -method
in recognizing bass drum sounds was more effective, compared
to snare drum sound recognition. The -method could suppress
undesirable harmonic components of the bass line which has
the large power in a low frequency band.
2) Snare Drum Sound Recognition: In group I, the f-measure
was drastically improved from 65.33% to 87.63% by enabling
both the -method and the -method. Table IV showed that the-method in recognizing snare drum sounds was less effective
than the -method.
In group II, on the other hand, the -method was more effective than the -method. These results suggest that the templateadaptation became to work correctly after suppressing harmoniccomponents in some pieces. In other words, the -method andthe -method helped each other in improving the f-measure, andthus it is important to use both methods.
In group III, however, the f-measure was slightly degraded byenabling the -method because the template-adaptation failed
in some pieces. In these pieces, the seed template was erroneously adapted to harmonic components. The -method was
not effective enough to recover from such erroneous adaptation.
These facts suggest that acoustic features of snare drum sounds
in these pieces are too different from those of the seed template.
To overcome these problems, we plan to incorporate multiple
templates for each drum instrument.
3) Hi-Hat Cymbal Sound Recognition: The f-measure in
hi-hat cymbal sound recognition (46.25% in total) was lowest
among the experimental results in recognizing three drum
instruments. The performance without the -method and the-method indicates that this is the most difficult task in our
experiments. Unfortunately, the -method was not effective
enough for hi-hat cymbals, while it reduced some errors asshown in Table IV. This is because there are three major playing
styles for hi-hat cymbals, closed, open, and half-open, and they
are used in a mixed way in an actual musical piece. Since ourmethod used just a single template, the template could notcover all spectral variations by those playing styles and was
not appropriately adapted to those sounds in the piece evenby the -method. We plan to incorporate multiple templates

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

343

TABLE IIIDRUM SOUND RECOGNITION RATES

Note: 70 musical pieces were sorted in descending order with respect to the f-measure by the fully-enabled procedure (i.e., SAM -procedure in bass andsnare drum sound recognition, AM -procedure in hi-hat cymbal sound recognition). The first 20 pieces were put in group I, and the next 25 ones were put
in group II, and the last 25 ones were put in group III.
TABLE IV
RECOGNITION ERROR REDUCTION RATES

Note: The definition of group I, II and III is described in Table III. This shows the recognition error reduction rates which represent the f-measure improvement
obtained by enabling the A-method added to the M -procedure, and that obtained by enabling the S -method added to the AM -procedure.
TABLE V
LIST OF MUSICAL PIECES SORTED IN DESCENDING ORDER WITH RESPECT TO f-MEASURE

Fig. 10. (a), (b): f-measure curves by three procedures in (a) bass drum sound recognition and (b) snare drum sound recognition along sorted musical pieces indescending order with respect to f-measure by SAM -procedure. (c): f-measure curves by two procedures in hi-hat cymbal sound recognition along sorted musicalpieces in descending order with respect to f-measure by AM -procedure.

as discussed above to deal with this difficulty while anotherproblem of identifying the playing styles of hi-hat cymbals will
still remain an open question.
VIII. CONCLUSION
In this paper, we have presented a drum sound recognition
system that can detect onset times of drum sounds and identify them. Our system used template-adaptation and templatematching methods to individually detect onset times of threedrum instruments, the bass drum, snare drum, and hi-hat cym-

bals. Since a drum-sound spectrogram prepared as a seed template is different from one used in a musical piece, our template-adaptation method adapts the template to the piece. By
using the adapted template, our template-matching method thendetects their onset times even if drum sounds are overlappedby other musical instrument sounds. In addition, to improve
the performance of the adaptation and matching, we proposeda harmonic-structure-suppression method that suppresses harmonic components of other musical instrument sounds by using
comb-filter-like spectral analysis.

344IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 1, JANUARY 2007

To evaluate our system, we performed reliable experimentswith popular-music CD recordings, which are the largest
experiments for drum sounds as far as we know. The experimental results showed that both of the template-adaptationand harmonic-structure-suppression methods improved thef-measure of recognizing each drum. The average f-measures
were 82.924%, 58.288%, and 46.249% in recognizing bass
drum sounds, snare drum sounds, and hi-hat cymbal sounds,
respectively. Our system, called AdaMast [33], in which theharmonic-structure-suppression method was disabled won thefirst prize of Audio Drum Detection Contest in MIREX2005.
We expect that these results could be used as a benchmark.
In the future, we plan to use multiple seed templates for eachkind of the drums to improve the coverage of the timbre variation of drum sounds. A study on timbre variation of drum sounds[34] seems to be helpful. The improvement of the templatematching method is also necessary to deal with the spectral variation among onsets. In addition, we will apply our system torhythm-related content description for building a content-based
MIR system.

REFERENCES[1] E. Scheirer, “Tempo and beat analysis of acoustic musical signals,” J.
Acoust. Soc. Am., vol. 103, no. 1, pp. 588–601, Jan. 1998.
[2] J. Paulus and A. Klapuri, “Measuring the similarity of rhythmic patterns,” in Proc. Int. Conf. Music Information Retrieval (ISMIR), 2002,
pp. 150–156.
[3] F. Gouyon and P. Herrera, “Determination of the meter of musicalaudio signals: seeking recurrences in beat segment descriptors,” inProc. Audio Engineering Soc. (AES), 114th Conv., 2003.
[4] E. Pampalk, S. Dixon, and G. Widmer, “Exploring music collections
by browsing different views,” J. Comput. Music J., vol. 28, no. 2, pp.
49–62, summer 2004.
[5] G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” IEEE Trans. Speech Audio Process., vol. 10, no. 5, pp. 293–302,
Jul. 2002.
[6] S. Dixon, E. Pampalk, and G. Widmer, “Classification of dance musicby periodicity patterns,” in Proc. Int. Conf. Music Information Retrieval(ISMIR), 2003, pp. 159–165.
[7] D. Ellis and J. Arroyo, “Eigenrhythms: Drum pattern basis sets for classification and generation,” in Proc. Int. Conf. Music Information Retrieval (ISMIR), 2004, pp. 554–559.
[8] C. Uhle and C. Dittmar, “Drum pattern based genre classification of
popular music,” in Proc. Int. Conf. Audio Eng. Soc. (AES), 2004.
[9] M. Goto and Y. Muraoka, “A sound source separation system for
percussion instruments,” IEICE Trans. D-II, vol. J77-D-II, no. 5, pp.
901–911, May 1994.
[10] P. Herrera, A. Yeterian, and F. Gouyon, “Automatic classification of
drum sounds: a comparison of feature selection methods and classification techniques,” in Proc. Int. Conf. Music and Artificial Intelligence(ICMAI), LNAI2445, 2002, pp. 69–80.
[11] J. Paulus and A. Klapuri, “Conventional and periodic N-grams in thetranscription of drum sequences,” in Proc. Int. Conf. Multimedia andExpo (ICME), 2003, pp. 737–740.
[12] ——, “Model-based event labeling in the transcription of percussive
audio signals,” in Proc. Int. Conf. Digital Audio Effects (DAFX), 2003,
pp. 73–77.
[13] O. Gillet and G. Richard, “Automatic transcription of drum loops,” inProc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2004, pp.
269–272.
[14] ——, “Drum track transcription of polyphonic music using noise
subspace projection,” in Proc. Int. Conf. Music Information Retrieval(ISMIR), 2005.
[15] M. Goto, “An audio-based real-time beat tracking system for musicwith or without drum-sounds,” J. New Music Res., vol. 30, no. 2, pp.
159–171, Jun. 2001.

[16] F. Gouyon, F. Pachet, and O. Delerue, “On the use of zero-crossing
rate for an application of classification of percussive sounds,” in Proc.
COST-G6 Conf. Digital Audio Effects (DAFX), 2000.
[17] A. Zils, F. Pachet, O. Delerue, and F. Gouyon, “Automatic extractionof drum tracks from polyphonic music signals,” in Proc. Int. Conf. Web
Delivering of Music (WEDELMUSIC), 2002, pp. 179–183.
[18] D. FitzGerald, E. Coyle, and B. Lawlor, “Sub-band independent subspace analysis for drum transcription,” in Proc. Int. Conf. Digital Audio
Effects (DAFX), 2002, pp. 65–69.
[19] C. Uhle, C. Dittmar, and T. Sporer, “Extraction of drum tracks frompolyphonic music using independent subspace analysis,” in Proc. Int.
Symp. Independent Component Analysis and Blind Signal Separation(ICA), 2003, pp. 843–848.
[20] J. Paulus and A. Klapuri, “Drum transcription with non-negative spectrogram factorisation,” in Proc. Eur. Signal Process. Conf. (EUSIPCO),
2005.
[21] T. Virtanen, “Sound source separation using sparse coding withtemporal continuity objective,” in Proc. Int. Computer Music Conf.
(ICMC), 2003, pp. 231–234.
[22] D. FitzGerald, B. Lawlor, and E. Coyle, “Prior subspace analysis fordrum transcription,” in Proc. Audio Eng. Soc. (AES), 114th Conv.,
2003.
[23] ——, “Drum transcription in the presence of pitched instruments using
prior subspace analysis,” in Proc. Irish Signals Syst. Conf. (ISSC),
2003, pp. 202–206.
[24] C. Dittmar and C. Uhle, “Further steps towards drum transcription ofpolyphonic music,” in Proc. Audio Eng. Soc. (AES), 116th Conv., 2004.
[25] A. Klapuri, “Sound onset detection by applying psychoacoustic knowledge,” in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP),
1999, pp. 3089–3092.
[26] P. Herrera, V. Sandvold, and F. Gouyon, “Percussion-related semantic
descriptors of music audio files,” in Proc. Int. Conf. Audio Eng. Soc.
(AES), 2004.
[27] V. Sandvold, F. Gouyon, and P. Herrera, “Percussion classification inpolyphonic audio recordings using localized sound models,” in Proc.
Int. Conf. Music Information Retrieval (ISMIR), 2004, pp. 537–540.
[28] A. Savitzky and M. Golay, “Smoothing and differentiation of data by
simplified least squares procedures,” J. Anal. Chem., vol. 36, no. 8, pp.
1627–1639, Jul. 1964.
[29] N. Otsu, “A threshold selection method from gray-level histograms,”
IEEE Trans. Syst., Man, Cybern., vol. SMC-6, no. 1, pp. 62–66, Jan.
1979.
[30] M. Goto, K. Itou, and S. Hayamizu, “A real-time filled pause detection system for spontaneous speech recognition,” in Proc. Eurospeech,
1999, pp. 227–230.
[31] M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, “RWC music database: popular, classical, and jazz music databases,” in Proc. Int. Conf.
Music Information Retrieval (ISMIR), 2002, pp. 287–288.
[32] ——, “RWC music database: music genre database and musical instrument sound database,” in Proc. Int. Conf. Music Information Retrieval(ISMIR), 2003, pp. 229–230.
[33] K. Yoshii, M. Goto, and H. Okuno, “AdaMast: a drum sound recognizer
based on adaptation and matching of spectrogram templates,” in Proc.
Music Information Retrieval Evaluation eXchange (MIREX), 2005.
[34] E. Pampalk, P. Hlavac, and P. Herrera, “Hierarchical organization andvisualization of drum sample libraries,” in Proc. Int. Conf. Digital
Audio Effects (DAFX), 2004, pp. 378–383.

Kazuyoshi Yoshii (S’05) received the B.S. and
M.S. degrees from Kyoto University, Kyoto, Japan,
in 2003 and 2005, respectively. He is currently
pursuing the Ph.D degree in the Department ofIntelligence Science and Technology, Graduate
School of Informatics, Kyoto University.
His research interests include music scene analysis
and human-machine interaction.
Mr. Yoshii is a member of the Information Processing Society of Japan (IPSJ) and Institute of Electronics, Information, and Communication Engineers(IEICE). He is supported by the JSPS Research Fellowships for Young Scientists (DC1). He has received several awards including the FIT2004 Paper Awardand the Best in Class Award of MIREX2005.

YOSHII et al.: DRUM SOUND RECOGNITION FOR POLYPHONIC AUDIO SIGNALS

345

Masataka Goto received the Doctor of Engineering
degree in electronics, information, and communication engineering from Waseda University, Tokyo,
Japan, in 1998.
He then joined the Electrotechnical Laboratory(ETL; reorganized as the National Institute of Advanced Industrial Science and Technology (AIST)in 2001), where he has been a Senior Research
Scientist since 2005. He served concurrently as a
Researcher in Precursory Research for Embryonic
Science and Technology (PRESTO), Japan Scienceand Technology Corporation (JST), from 2000 to 2003, and an Associate
Professor in the Department of Intelligent Interaction Technologies, Graduate
School of Systems and Information Engineering, University of Tsukuba,
Tsukuba, Japan, since 2005. His research interests include music information
processing and spoken language processing.
Dr. Goto is a member of the Information Processing Society of Japan (IPSJ),
Acoustical Society of Japan (ASJ), Japanese Society for Music Perception andCognition (JSMPC), Institute of Electronics, Information, and Communication
Engineers (IEICE), and International Speech Communication Association(ISCA). He has received 18 awards, including the IPSJ Best Paper Award and
IPSJ Yamashita SIG Research Awards (special interest group on music andcomputer, and spoken language processing) from the IPSJ, the Awaya Prize for
Outstanding Presentation and Award for Outstanding Poster Presentation fromthe ASJ, Award for Best Presentation from the JSMPC, Best Paper Award forYoung Researchers from the Kansai-Section Joint Convention of Institutes ofElectrical Engineering, WISS 2000 Best Paper Award and Best Presentation
Award, and Interaction 2003 Best Paper Award.

Hiroshi G. Okuno (SM’06) received the B.A. and
Ph.D degrees from the University of Tokyo, Tokyo,
Japan, in 1972 and 1996, respectively.
He worked for Nippon Telegraph and Telephone,
Kitano Symbiotic Systems Project, and Tokyo University of Science. He is currently a Professor in the
Department of Intelligence Science and Technology,
Graduate School of Informatics, Kyoto University,
Kyoto, Japan. He was a Visiting Scholar at Stanford
University, Stanford, CA, and Visiting Associate Professor at the University of Tokyo. He has done research in programming languages, parallel processing, and reasoning mechanisms in AI, and is currently engaged in computational auditory scene analysis,music scene analysis, and robot audition. He edited (with D. Rosenthal) Computational Auditory Scene Analysis (Princeton, NJ: Lawrence Erlbaum, 1998) and(with T. Yuasa) Advanced Lisp Technology (London, U.K.: Taylor &Francis,
2002).
Dr. Okuno has received various awards including the 1990 Best Paper Awardof JSAI, the Best Paper Award of IEA/AIE-2001 and 2005, and IEEE/RSJ
Nakamura Award for IROS-2001 Best Paper Nomination Finalist. He was also
awarded 2003 Funai Information Science Achievement Award. He is a memberof the IPSJ, JSAI, JSSST, JSCS, RSJ, ACM, AAAI, ASA, and ISCA.