SINGER IDENTIFICATION BASED ON ACCOMPANIMENT SOUND
REDUCTION AND RELIABLE FRAME SELECTION
Hiromasa Fujihara,† Tetsuro Kitahara,† Masataka Goto,‡
Kazunori Komatani,† Tetsuya Ogata,† and Hiroshi G. Okuno†† Dept. of Intelligence Science and Technology

Graduate School of Informatics, Kyoto University
Sakyo-ku, Kyoto 606-8501, Japan{fujihara, kitahara, komatani, ogata, okuno}@kuis.kyoto-u.ac.jp

ABSTRACTThis paper describes a method for automatic singer identification from polyphonic musical audio signals including sounds of various instruments. Because singing voices
play an important role in musical pieces with a vocal part,the identification of singer names is useful for music information retrieval systems. The main problem in automatically identifying singers is the negative influences causedby accompaniment sounds. To solve this problem, wedeveloped two methods, accompaniment sound reduction
and reliable frame selection. The former method makes it
possible to identify the singer of a singing voice after reducing accompaniment sounds. It first extracts harmoniccomponents of the predominant melody from sound mixtures and then resynthesizes the melody by using a sinusoidal model driven by those components. The lattermethod then judges whether each frame of the obtained
melody is reliable (i.e. little influenced by accompanimentsound) or not by using two Gaussian mixture models for
vocal and non-vocal frames. It enables the singer identification using only reliable vocal portions of musical pieces.
Experimental results with forty popular-music songs by
ten singers showed that our method was able to reducethe influences of accompaniment sounds and achieved an
accuracy of 95%, while the accuracy for a conventional
method was 53%.
Keywords: Singer identification, artist identification,
melody extraction, singing detection, similarity-based
MIR

1

INTRODUCTION

Singing voice is known as the oldest musical instrumentthat everyone has by nature and plays an important role inmany musical genres, especially in popular music. When
Permission to make digital or hard copies of all or part of thiswork for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the fullcitation on the first page.
c
2005
Queen Mary, University of London‡ National Institute of Advanced Industrial

Science and Technology (AIST)
Tsukuba, Ibaraki 305-8568, Japanm.goto@aist.go.jp

a song is heard, for example, most people use the vocal
part by the lead singer’s voice as a primary cue for recognizing (identifying) the song (name). Therefore, mostmusic stores classify music according to singers’ names(often referred to as artist names) in addition to musicalgenres.
As known from the above importance of the singing
voice, the description of singer names of songs is useful for music information retrieval (MIR). When a user
wants to find songs sung by a certain singer, an MIR system can use the description of singer names (artist names).
Furthermore, detailed descriptions of acoustical characteristics of singing voices can also play an important rolein MIR because they are useful for computing acousticalsimilarities between singers. Most previous MIR systems,
however, assumed that the metadata including artist names
and song titles were available: if they were not available
for some songs, those songs cannot be retrieved by submitting a query of their artist names.
To achieve such singing-voice-based MIR and compute artist similarities without requiring the metadata forevery song to be prepared, in this paper, we focus on
the problem of identifying singers for songs, automatic
singer identification problem. This problem is difficult
because most singing voices are accompanied by othermusical instruments. It is therefore necessary to focus
on the vocal part in polyphonic sound mixtures whileconsidering the negative influences from accompanimentsounds. In other words, feature vectors extracted frommusical audio signals are influenced by the sounds ofaccompanying instruments. Although speaker identification problem for (non-music) speech signals has been
dealt with by many studies in the field of speech information processing, their results cannot be directly applied
to the singer identification problem for singing voiceswith accompaniments because most existing speakeridentification techniques assume speech signals presented
without other simultaneous sounds. On the other hand,
Tsai et al. [1, 2] have pointed out this problem and have
tried to solve it by using a statistically-based speakeridentification method for speech signals in noisy environments [3]. On the assumption that singing voices and accompaniment sounds are statistically independent, theyfirst estimated an accompaniment-only model from interlude sections and a vocal-plus-accompaniment model329

from whole songs, and the estimated a vocal-only modelby subtracting the accompaniment-only model from thevocal-plus-accompaniment model. However, this assumption is not always satisfied and the way of estimating
the accompaniment-only model has a problem: accompaniments during vocal sections and performances (accompaniments) during interlude sections can have different acoustical characteristics. In other previous studies [4, 5, 6, 7], the accompaniment sound problem has not
explicitly been dealt with.
To solve this problem, we propose two methods: accompaniment sound reduction and reliable frame selection. Using the former method, we reduce the influence ofaccompaniment. We first extracted the harmonic structureof the melody from audio signals, and then, resynthesize itusing a sinusoidal model. This method reduces the influence of accompaniment sounds. The latter method selects
frames that are reliable enough for classification.
The rest of this paper is organized as follows. In thenext section, we describe our method for a singer identification task. In Section 3, we describe the implementationof our system. In Section 4, we describe our experiments
and present the results. In Section 5, we draw conclusions
and point out future directions.

F0 Estimation

Harmonic Structure Extraction

Resynthesis

Feature Extraction

Vocal
GMM
Non-Vocal
GMM

thresholdlikelihood>η
No
Yes
AcceptSinger Determination

This paper describes an automatic singer identification
system, which is the system for determining a singer’sname of given musical audio signals. The target data
are real-world musical audio signals such as popular music CD recordings that contain singing voices of a single
singer and accompaniment sounds.
The main difficulty in achieving automatic singer
identification lies in the negative influences of accompaniment sounds. Since a singing voice usually exists withaccompaniment sounds at the same time, acoustical features that are extracted from such a singing voice will bedependent on the accompaniment sounds. When featuresthat are commonly used in speaker identification studies,such as cepstral coefficients or linear prediction coefficients (LPC), are extracted, in fact, those to be obtained
from musical audio signals will represent not solely thesinging voice but a mixture of the singing voice and theaccompaniment sounds. To achieve accurate singer identification, therefore, it is indispensable to cope with thisaccompaniment sound problem.
One possible solution to this problem may be to use
data influenced by accompaniment sounds for both training and identification. In fact, most of the previous studies [4, 5, 6, 7] adopted this approach. However, it often
fails because accompaniment sounds usually have different acoustical features from song to song. For example,
the acoustical similarity for two musical pieces, the accompaniments of which are on a piano solo and a fullband, respectively, will not become high enough, even if
they are sung by the same singer.
To solve the problem, we developed two methods: ac-

Output (Singer’s name)

RejectReliable Frame
Selectionfor each frame

SINGER IDENTIFICATION ROBUSTTO ACCOMPANIMENT SOUNDS

330

Accompaniment Sound
Reduction

2

Input (Audio signals)

Figure 1: Method overview
companiment sound reduction and reliable frame selection. Figure 1 shows an overview of our methods.
2.1

Accompaniment Sound Reduction

One of the best solutions to the accompaniment soundinfluence is to reduce the accompaniment sounds froma given audio signal. In order to achieve this, we use
a melody resynthesis technique based on the harmonic
structure that consists of the following three parts:
1. Estimating the fundamental frequency (F0) of the
melody using Goto’s PreFEst [8].
2. Extracting the harmonic structure corresponding to
the melody.
3. Resynthesizing the audio signal (waveform) corresponding to the melody using a sinusoidal synthesis.
Thus, we obtain the waveform corresponding only to the
melody. Note that the melody’s waveform obtained withthis method contains instrument (i.e., non-vocal) sounds
in interlude sections as well as voices in singing sections,because the melody is just defined as the most predominant note in each frame [8]. It may therefore be considerednecessary to detect singing sections. In practice, however,we can omit this detection, which is a difficult problem,by using reliable frame selection described below.
2.2

Reliable Frame Selection

Another solution to the accompaniment sound influence
is to select frames that are less influenced by the accompaniment sounds and to use only them for identification.
We call this approach reliable frame selection. In orderto achieve this, we introduce two kinds of Gaussian mixture models (GMMs), a vocal GMM λV and a non-vocal
GMM λN . The vocal GMM λV is trained on feature vectors extracted from singing sections, and the non-vocal
GMM λN is trained on those extracted from interlude sections. Given a feature vector x, the likelihoods for thetwo GMMs, p(x|λV ) and p(x|λN ), represent how the feature vector x is like a vocal or a (non-vocal) instrument,
respectively. If the feature vector x is less influenced byaccompaniment sounds (i.e., more reliable), p(x|λV ) will
be higher and p(x|λN ) will be lower. We therefore determine whether the feature vector x is reliable or not basedon the following equation:
reliablelog p(x|λV ) − log p(x|λN )

≥
<

η,

(1)

not-reliable

where η is a threshold. In our experiments, we use 64mixture GMMs. It is difficult to decide a universal threshold for a variety of songs because we cannot select enoughfeature vectors for classification from a song which have
few reliable frames. We therefore determine the threshold dependent on songs so that α % of the whole frames
in the song are selected as reliable frames. Note that mostof the non-vocal frames are rejected in this selection step.
This means that we can avoid detecting singing sections
by using this reliable frame selection.

3

IMPLEMENTATIONIn this section, we describe the implementation of our system. As described above, our system consists of the following four phases: accompaniment sound reduction, feature extraction, reliable frame selection and classification.

in frequency-range-limited sound mixtures. Since themelody line tends to have the most predominant harmonic
structure in middle- and high-frequency regions, we canestimate the F0s of the melody by applying PreFEst withreliable frequency-range limitation.
We will describe a summary of PreFEst below. Hereafter, x is the log-scale frequency denoted in units ofcents (a musical-interval measurement), and (t) means(t)time. Given the power spectrum Ψ p (x), we first apply
a band-pass filter (BPF) that is designed so that it covers most of the dominant harmonics of typical melody
lines. The filtered frequency components can be repre(t)sented as BPF(x)Ψ p (x), where BPF(x) is the BPF’s frequency response for the melody line. To enable the application of statistical methods, we represent each of the
bandpass-filtered frequency components as a probability(t)density function (PDF), called an observed PDF, pΨ (x):
(t)

(t)

pΨ (x) = 

BPF(x)Ψ p (x)(t)∞−∞ BPF(x)Ψ p (x)dx(2).

Then, we consider each observed PDF to have been generated from a weighted-mixture model of the tone modelsof all the possible F0s, which is represented as follows:p(x|θ (t) ) = Fhw(t) (F)p(x|F)dF(3)θ (t) = {w (F)|Fl ≤ F ≤ Fh},

(4)

Fl
(t)

where p(x|F) is the PDF of the tone model for each F0,
and Fh and Fl is defined as lower and upper limits of thepossible (allowable) F0 range, and w(t) (F) is the weightof a tone model that satisfies
 Fli
Fhi

w(t) (F)dF = 1.

(5)

Tone model represents a typical harmonic structrue and indicates where the harmonics of the F0 tend to occur. Then,we estimate w(t) (F) using EM algorithm and regard it as
the F0’s PDF. Finally, we obtain the most dominant F0s
F (t) by the following equation:
F (t) = argmax w(t) (F)

3.1

Pre-Processing

Given an audio signal, it is monauralized and downsampled to 16 kHz. Then, the spectrogram is calculatedusing the short-time Fourier transform shifted by 10.0 ms(160 points) with a 2048-point (128.0 ms) Hamming window.
3.2

Accompaniment Sound Reduction

Using the method described in Section 2.1, we reduce accompaniment sounds as follows:
3.2.1

3.2.2

Harmonic Structure Extraction

Based on the estimated F0, we extract the power and thephase of fundamental frequency component and harmoniccomponents. For each component, we allow |r| cent error
and extract the peak in the allowed area. The power Al , thephase θl and frequency Fl of l th overtone (l = 1, . . . , 20)
can be represented asFl= argmax |S(F)|Fr

F0 Estimation

We use Goto’s PreFEst [8] for estimating the F0s of the
melody. PreFEst estimates the most predominant F0

(6)

F

Al
θl

r

(lF · (1 − 2 1200 ) ≤ F ≤ lF · (1 + 2 1200 )), (7)= |S(Fl )|,(8)= arg S(Fl ),
(9)

331

linear power
0

2000

4000

6000

8000

Frequency [Hz]linear power(a) An original spectrum and its envelope.

order to compute cepstral coefficients [10], we take thelog-magnitude descrete cosine transform (DCT) from thepower spectrum. The envelops are represented in lower
order of the cepstral coefficients, while the fine structures
are in higher order. Mel-frequency is a logarithmic frequency scale fitted to the characteristics of the human auditory sense. For the MFCC computations, mel-filterbank
analysis is applied first. Then, we obtain the MFCC
from the log-magnitude DCT. In this paper, we use 15
dimensional MFCC, caluculated via 20 dimensional melfilterbank analysis.
3.3.2

0

2000

4000
6000
Frequency [Hz]8000(b) An extracted spectrum and its envelope.
Figure 2: Example of F0 estimation and harmonic structure extraction. Envelopes of the spectrums are calculatedusing linear prediction (LP) analyses.

Linear Prediction Coefficients (LPC)

Linear prediction (LP) analysis [13, 14] is a method for
estimating the transfer function of vocal tract, assumingthat input audio signal contains only human voice. In the
LP model, given a signal s(n), we predict the signal as alinear combination of its previous samples. The predictedvalue sW (n) is given bypsW (n) = ∑ αi sW (n − i) + g(n),

(11)

i=1

where S(F) denotes spectrum, F denotes F0 estimated by
the PreFEst. In our experiments, we set r to 20.
Figure 2 shows an example of F0 estimation and harmonic structure extraction. Figure 2 (a) shows an original
spectrum and its envelope and Figure 2 (b) shows an extracted spectrum and its envelope. As seen in the figures,a spectral envelope of extracted spectrum precisely represents formants of singing voice, compared with that oforiginal spectrum.
3.2.3

Resynthesis

We resynthesize the audio signals of the melody fromthe extracted harmonic structure by using a sinusoidal
model [9]. Resynthesized audio signals are expressed asLs(t) = ∑ Al cos(ωl t + θl ),

(10)

l=1

where Al , θl , Fl represent the power, the phase and thefrequency of the l th overtone and t is time.
3.3

Feature Extraction

We calculate feature vectors from the resynthesized audio
signals. It is known that the individual characteristics ofspeech signals are expressed in their spectral envelopes.
In the field of speech recognition studies, in fact, various
methods have been proposed [10] for calculating feature
vectors concerning spectral envelopes. Here, we compare
some of them, which are commonly used in speech recognition studies.
3.3.1

Mel-frequency Cepstral Coefficients (MFCC)

MFCCs [11, 12] are cepstral coefficients calculated on amel-frequency scale. Cepstral analysis is the method toseparate envelope of spectrum from fine structure. In

332

where p represents the order of the predictor, αi s are defined as the linear prediction coefficients (LPC), and g(n)represents the error in the model. The LPCs are determined by minimizing the mean squared prediction errorof g(n). We use 20th-order LPC in this paper.
3.3.3

LP-derived Cepstral Coefficients (LPCC)

LPCCs [13] are cepstral coefficients of a LPC spectrum.
Cepstral analysis on the LPC spectrum plays a role of orthogonalization and is known to be effective in patternrecognition. The LPCCs c(n) is directly obtained fromthe LPC with the following equation:⎧log σ 2
⎪
⎪

⎪
n−1 
⎪
k
⎪
⎨ α +
n
∑ 1 − c(k)αn−kc(n) =
k=1  n

⎪
⎪
n−1
⎪
k
⎪
⎪
c(k)αn−k⎩ ∑ 1−nk=1(n = 0)(1 ≤ n ≤ p),(n > p)(12)

where σ 2 represents the power of the signal, αn represent
the LPCs, and p represents an order of the LPC. We set
the order of the LPCC to 15 in this paper.
3.3.4

Linear Prediction Mel Cepstral Coefficients(LPMCC)LPMCCs are mel-cepstral coefficients of LPC spectrum.
In addition to the role of orthogonalization, the LPMCCs
are superior to the LPC in terms of suitability to the human
auditory sense, which is a benefit of the mel-frequency
scale. We derive the LPMCC by computing the MFCC
from the LPC spectrum because of simplicity of implementation. We set the order of the LPMCC to 15 in thispaper.

Table 1: Training data for reliable frame selection.
Name
Shingo Katsuta
Yoshinori Hatae
Masaki Kuehara
Hiroshi Sekiya
Katsuyuki Ozawa
Masashi Hashimoto
Satoshi Kumasaka
OrikenKonbu
Eri Ichikawa
Tomoko Nitta
Kaburagi Akiko
Yuzu Iijima
Reiko Sato
Tamako Matsuzaka
Donna Burke

Gender
M
M
M
M
M
M
M
M
F
F
F
F
F
F
F
F

Piece Number
027
037
032, 078
048, 049, 051
015, 041
056, 057
047
006
013
020
026
055
060
063
070
081, 089, 091, 093, 097

Table 2: Songs used for evaluation. The numbers written
in the table are piece numbers of RWC-MDB-P-2001.
a
b
c
d
e
f
g
h
i
j

3.4

Name
GenderKazuo Nishi
M
Hisayoshi Kazato
M
Kousuke Morimoto
M
Shinya Iguchi
M
Jeff Manning
M
Hiromi Yoshii
F
Tomomi Ogata
F
Rin
F
Makiko Hattori
F
Betty
F

D1
012
004
038
082
085
002
007
014
065
086

D2
029
011
039
084
087
017
028
021
067
092

D3
036
019
042
088
095
069
052
050
068
094

D4
043
024
044
090
098
075
080
053
077
096

Reliable Frame Selection

We select frames that are reliable and influenced a little byaccompaniment sounds based on the method described in
Section 2.2.
3.5

Singer Determination

The name of the singer is determined based on 64-mixture
GMMs. Let X = {xt |t = 1, . . . , T } be a time series of feature vectors selected in the reliable frame selection phase,and λs be the GMM for the singer s. Then, the name ofthe singer is determined through the following equation:s = argmax
i

4

1 T
∑ log p(xt |λi ).
T t=1

(13)

EXPERIMENTS

In this section, we describe the experiments that were conducted to evaluate our system.
4.1

Effectiveness of the whole system

To confirm the effectiveness of our methods, accompaniment sound reduction and reliable frame selection, we
conducted experiments on singer identification under thefollowing four conditions:

Table 3: Experimental results that show effectiveness of
the whole system, where “reduc.” and “selec.” mean accompaniment sound reduction and reliable frame selection, respectively.
a
b
c
d
e
f
g
h
i
j
Total

(i)
baseline
1/4
3/4
2/4
4/4
1/4
1/4
0/4
4/4
4/4
1/4
53%

(ii)
reduc. only
2/4
1/4
2/4
4/4
0/4
2/4
2/4
4/4
4/4
3/4
60%

(iii)
selec. only
2/4
3/4
3/4
4/4
0/4
2/4
0/4
4/4
3/4
2/4
58%

(iv)
ours
4/4
4/4
4/4
4/4
4/4
3/4
3/4
4/4
4/4
4/4
95%

(i) without both the reduction and the selection (baseline),(ii) without the reduction, with the selection,(iii) with the reduction, without the selection, and(iv) with both the reduction and the selection (ours).
We used forty songs by ten different singers (five weremale and five were female), listed in Table 2, taken from
“RWC Music Database: Popular” [15]. Using these data,
we conducted the 4-fold cross validation, that is, we firstdivided the whole data into four groups, Di (i = 1, 2, 3, 4)
in Table 2, and then repeated the following step four times:
each time, we left out one of the four groups for training
and used the omitted one for testing. As the training data
for the reliable frame selection, we used twenty-five songsof sixteen different singers listed in Table 1, also takenfrom “RWC Music Data: Popular”, which differ from thesingers used for evaluation. We set α to 15%, in reference
to the experiment described in Section 4.2. As a feature
vector, in response to the experiment described in Section
4.3, we use the LPMCC with the reduction and the MFCC
without. We adopt the MFCC for the experiment without the reduction, because, as described in Section 3.3.2,
the LPMCC that is based on LPC can be applied only tohuman voice.
Table 3 shows results of the experiments. As seenin the table, accompaniment sound reduction and reliable
frames selection improved the accuracy of singer identification. When these two methods were used together, inparticular, the accuracy was significantly improved: from
53% to 95%.
Figure 3 shows confusion matrices of the experiments.
As seen in the figure, confusions between male and female
decreased by using the reduction method. It means that, inthe cases of (ii) and (iv), the reduction method reduced theinfluences of accompaniment sound, and the system could
correctly identify the genders. On the other hand, in thecases without the reduction method (Conditions (i) and(iii)), the influences of accompaniment sound prevented
the system from correctly identifying even the genders of
the singers.

333

i

classification result

j

a
b
c
d
e
f
g
h
i
j

a b c d e

f

g h

i

a
b
c
d
e
f
g
h
i
j

(i) baselineclassification resultja b c d e(ii) reduction only

f

g h

i

classification resulta b c d e

j

a
b
c
d
e
f
g
h
i
j

singer’s labelg hsinger’s label

singer’s labelfsinger’s label

classification resulta b c d e(iii) selection only

f

g h

i

j

a
b
c
d
e
f
g
h
i
j

(iv) both reduction and selection

Figure 3: Confusion matrices. Center lines in each figure are boundaries between male and female. Note that confusion
between male and female decreased by using the accompaniment sound reduction method.

100

Accuracy (%)

90
80
70
60
50

reduction only
reduction and selection

40
30
5

10

15

20

25

30

α (%)

Figure 4: Experimental results that show dependency of
accuracy on α . α % of all frames was judged as reliable.
4.2

Dependency of accuracy on α

We conducted experiments with setting α to various values to investigate the dependency of accuracies on α . Experimental results shown in Figure 4 show that classification accuracy was not affected by small changes of α . It
is also noticeable that a value of α that gives the highest
accuracy differed. The reason of this fact is the following: accompaniment sound reduction method reduced theinfluences of accompaniment sounds and emphasized thedifferences between reliable and unreliable frames. Thus,
if we raised α too much, the system selected many unreliable frames and the system performance decreases. Without the reduction method, on the other hand, reliability ofeach frame did not make much difference because of theinfluences of accompaniment sounds. Therefore, it was
profitable to use many frames for classification by setting
α comparatively higher. However, in this case, we could
not attain sufficient classification accuracy because of theinfluences of accompaniment sounds.
4.3
4.3.1

Investigation of Accompaniment Sound
Influence and Comparison of Features
Conditions

There are two purposes in the experiments here. The
first one is to investigate the influence by accompanimentsounds. We investigated it by comparing our results to334

Table 4: Investigation of accompaniment sound influenceand comparison of features. The feature name written
in bold font is the one that gives the highest accuracy.
“corr. F0s” and “est. F0s” mean using correct F0s andestimated F0s, respectively, and “reduc.” means accompaniment sound reduction.

MFCC
LPC
LPCC
LPMCC

(i)
(ii)
Using vocal-only
w/o
with
reduc.
reduc.
98%
95%
83%
88%
95%
98%
98%
98%

(iii)
(iv)
(v)
Using mixed-down datawith reduc.
w/ocorr. F0s est. F0s reduc.
78%
75%
75%
50%
58%
48%
75 %
75%
63%
88%
83%
68%

ones for vocal-only data. In addition, we compared estimating F0s of melodies and using given correct ones. In
our experiments, we virtually generated the correct F0s by
estimating ones using vocal-only tracks. Although the estimates using vocal-only tracks were not completely correct, its accuracy was sufficiently high comparing withestimating ones using mixed-down versions. The second
purpose is to compare a variety of features. We used fourkinds of features described in Section 3.3 and comparedthese results. In the experiments here, we manually cut out
60 seconds of singing regions for each song, because weomitted reliable frame selection method in order to investigate the effectiveness of the only accompaniment soundreduction method.
4.3.2

Results and Discussion

Table 4 shows the results of the experiments. When we focused on the differences between the cases (iv) and (v), theaccuracies for the LPC, the LPCC, and the LPMCC were
improved by the reduction method, whereas that for the
MFCC was not improved. This is because the LPC etc.
assume that given signal contains only a single speech.
For this assumption, the accuracies for the LPC etc. in thecase (v) were low because the inputs were a mixture of
singing voices and accompaniment sounds. Because thecase (iv) dealt with signals obtained by extracting only
singing voices, the accuracies were higher than the case(v). Because the MFCC is not based on such an assump-tion, on the other hand, the accuracy for the MFCC in
the case (v) was high, but that in the case (iv) was same.
Because the LPMCC models the sounding mechanism of
humans’ voices, it could be expected to achieve a high accuracy if the assumption is satisfied. In fact, whereas theaccuracy for the LPMCC was lower than that for MFCC
in the case (v), that for the LPMCC was higher than thatfor the MFCC in the case (iv). Whereas most previous
studies used the MFCC, we achieved to adopt more robust features by reducing accompaniment sounds.
When we compared the LPC, the LPCC, and the
LPMCC, the accuracy for the LPCC was 17% higher thanthat for the LPC, and that for the LPMCC was 8% higher
than that for the LPCC. The reason why the accuracy for
the LPCC was higher than that for the LPC is that the
LPCCs are orthogonal features unlike the LPC. The reason why the accuracy for the LPMCC was higher than thatfor the LPCC is mel-frequency cepstrum allow better suppression of insignificant spectral variation in the higher
frequency bands.
The accuracies for the cases (iii) and (iv) were comparatively close. This was because partial misestimationof F0s was not critically connected to errors for singer
identification since the names of singers were determinedbased on the mean of a time series of likelihoods.
The case (iv) in experiment described in Section 4.1
was superior to the case (iv) in this experiment, eventhough we manually fed singing regions into the system
for this experiment. This means that the selection method
actually functioned not only as distinguishing vocal andnon-vocal frames but also as determining whether eachframe was reliable or not. The case (v) in this experiment,
however, was inferior to the case (iii) in experiments described in 4.1. This result means that, to accurately selectreliable frames, it is indispensable to use both the reduction method and the selection method together.
Table 5 lists an excerpt of experimental results foreach singer. As seen in the table, the reduction method
improved accuracies particularly for the singer (g). Thiswas because the songs of the singer (g) have different
kinds of genres such as a piano ballad and R&B. Thesesongs are accompanied on different instruments and hence
have different acoustical characteristics of accompaniment sounds. Whereas the system without the reduction
method did not correctly identify the singer’s name forthese songs, that with our method did. This result showsthat the reduction method could reduce the influence of
acoustical differences in accompaniment sounds. In contrast, identification errors for the singer (e) increased by
the reduction method. This is because the melodies’ F0s
were incorrectly estimated in some songs. We can also
confirm this by the fact that identification errors did notincrease when we provided the correct F0s.

5

CONCLUSION

We have described two methods that work in combination
to automatically identify singers for music information retrieval. To identify the singer names of musical pieces including sounds of various instruments, our method solved

Table 5: Accuracy for each singer, where “reduc.” means
accompaniment sound reduction.

a
b
c
d
e
f
g
h
i
j
Total

(iii)
with reduc.,corr. F0s
LPMCC
3/4
4/4
4/4
4/4
3/4
3/4
2/4
4/4
4/4
4/4
88%

(iv)
with reduc.,
est. F0s
LPMCC
3/4
4/4
4/4
4/4
2/4
2/4
2/4
4/4
4/4
4/4
83%

(v)
w/o
reduc.
MFCC
3/4
4/4
3/4
4/4
3/4
2/4
0/4
4/4
4/4
3/4
75%

the problem of the accompaniment sound influences. In
our experiments with forty songs by ten singers, we foundthat our methods achieved identification accuracy of 95%
and confirmed the robustness and effectiveness of thosemethods.
The main contributions of this paper can be summarized as follows:• We clarified the problem of the accompanimentsound influence for singer identification, which has
not been dealt with except for only a few attempts,
and provided two effective solutions, accompaniment sound reduction and reliable frame selection.
• The use of the accompaniment sound reduction
method made it possible to reduce the negative influence of accompaniment sound by extracting andresynthesizing the harmonic structure of the predominant melody. Though similar methods have beenused to improve the noise robustness in the field ofspeech recognition [16], this is the first paper thatshows its effectiveness for singer identification.
• The reliable frame selection method made it possible to select frames reliable enough for classification. Although similar methods were used in previous studies, they focused on distinguishing vocal
and non-vocal frames; they did not consider the reliability of each frame. Note that our selection method
rejects even unreliable vocal frames as well as nonvocal frames.
• We showed an investigation of features for singer
identification. While many features have been proposed in the field of speech recognition [10], it has
not been clear which feature was appropriate forsinger identification. We compared various featuresin terms of the singer identification, and found that
the LPMCC was the most robust among them. This
result will contribute to the progression of singer
identification research.
In the future, we plan to extend our method to calculate acoustical similarities between musical pieces in335

terms of singers and apply it to music information retrieval
based on singing voice similarities.

ACKNOWLEDGEMENTS
This research was partially supported by the Ministryof Education, Culture, Sports, Science and Technology (MEXT), Grant-in-Aid for Scientific Research (A),No.15200015, and Informatics Research Center for Development of Knowledge Society Infrastructure (COEprogram of MEXT, Japan). We thank everyone who has
contributed to building and distributing the RWC Music Database [15]. We also thank Kazuyoshi Yoshii and
Takuya Yoshioka (Kyoto University) for their valuable
discussions.

REFERENCES[1] Wei-Ho Tsai, Hsin-Min Wang, and Dwight Rodgers.
Automatic singer identification of popular music
recordings via estimation and modeling of solo vocal signal. In Proceedings of European Conference on Speech Communication and Technology(Eurospeech2003), 2003.
[2] Wei-Ho Tsai and Hsin-Min Wang. Automatic detection and tracking of target singer in multi-singer
music recordings. In Proceedings of the 2004 IEEE
International Conference on Acoustics, Speech, andSignal Processing (ICASSP 2004), pages 221–224,
2004.

for detecting melody and bass lines in real-world audio signals. Speech Communication, 43(4):311–329,
2004.
[9] James Anderson Moorer. Signal procesing aspectsof computer music: A survey. Proceedings of theIEEE, 65(8):1108–1137, 1977.
[10] Joseph Picone. Signal modeling techniques inspeech recognition. IEEE Proceedings, 81(9):1215–
1247, 1993.
[11] Steven B. Davis and Paul Mermelstein. Comparisonof parametric representation for monosyllabic word
recognition. IEEE Transactions on Acoustic, Speechand Signal Processing, 28(4):357–366, 1980.
[12] Beth Logan. Mel frequency cepstral coefficients formusic modelling. In Proceedings of the International Symposium on Music Information Retrieval(ISMIR 2000), pages 23–25, 2000.
[13] Bishnu S. Atal. Effectiveness of linear prediction
characteristics of the speech wave for automatic
speaker identification and verification. the Journalof the Acoustical Society of America, 55(6):1304–
1312, 1974.
[14] K. Shikano. Evaluation of LPC spectral matchingmeasures for phonetic unit recognition. Technical
Report CMU-CS-96-108, CMU, Computer Science
Department, 1986.

[3] R. C. Rose, E. M. Hofstetter, and D. A. Reynolds.
Integrated models of signal and background with application to speaker identification in noise. IEEE
Transactions on Speech and Audio Processing,2(2):245–257, 1994.

[15] Masataka Goto, Hiroki Hashiguchi, Takuichi
Nishimura, and Ryuichi Oka. RWC music database:
Popular, classical, and jazz music databases. In Proceedings of the 3rd International Conference on Music Information Retrieval (ISMIR 2002), pages 287–
288, October 2002.

[4] Brian Whitman, Gary Flake, and Steve Lawrence.
Artist detection in music with Minnowmatch. In
Proceedings of the 2001 IEEE Workshop on Neural Networks for Signal Processing, pages 559–568,
2001.

[16] Tomohiro Nakatani and Hiroshi G. Okuno. Harmonic sound stream segregation using localizationand its application to speech stream segregation.
Speech Communcations, 27:209–222, 1999.

[5] Adam L. Berenzweig, Daniel P. W. Ellis, and Steve
Lawrence. Using voice segments to improve artist
classification of music. In AES 22nd International
Conference on Virtual, Synthetic, and Entertainment
Audio, 2002.
[6] Youngmoo Edmund Kim and Brian Whitman.
Singer identificatin in popular music recordings using voice coding features. In Proceedings of the 3rdInternational Conference on Music Information Retrieval (ISMIR2002), pages 164–169, 2002.
[7] Tong Zhang. Automatic singer identification. In Proceedings of IEEE International Conference on Multimedia & Expo (ICME 2003), 2003.
[8] Masataka Goto.
description system:336

A real-time music-scenepredominant-F0 estimation