➠➡
AN AUTO-REGRESSIVE, NON-STATIONARY EXCITED SIGNAL PARAMETERESTIMATION METHOD AND AN EVALUATION OF A SINGING-VOICE RECOGNITION
Akira Sasou1 , Masataka Goto1, Satoru Hayamizu2 , Kazuyo Tanaka3
1

National Institute of Advanced Industrial Science and Technology (AIST)
2{a-sasou,m.goto}@aist.go.jp

Faculty of Engineering, Gifu University
hayamizu@cc.gifu-u.ac.jp

3Institute of Library and Information Science, University of Tsukuba
ktanaka@ulis.ac.jp

ABSTRACT
We have previously described an Auto-Regressive Hidden
Markov Model (AR-HMM) and an accompanying parameter estimation method. The AR-HMM was obtained by
combining an AR process with an HMM introduced as anon-stationary excitation model. We demonstrated that the
AR-HMM can accurately estimate the characteristics of both
articulatory systems and excitation signals from high-pitchedspeech. In this paper, we apply the AR-HMM to feature extraction from singing voices and evaluate the recognition
accuracy of the AR-HMM-based approach.

[6], the Mel-Frequency Cepstral Coefﬁcient(MFCC) was
adopted as a speech feature. In the frequency domain, it
is difﬁcult for the MFCC to retain the formant informationfor each sound as is the case with the LPC because harmoniccomponents of high-pitched speech become sparse. In orderto overcome the difﬁculties of singing-voice recognition, weapplied the speech analysis method based on the AR-HMM.
Using the AR-HMM-based method, we extracted the features of the singing voices and evaluated the recognition accuracy.
2. AUTO-REGRESSIVE HIDDEN MARKOV
MODEL

1. INTRODUCTION
The linear prediction (LP) method is widely used for the
analysis of speech signals [1, 2]. However, several problems
remain to be resolved. For example, (1) local peaks of LP
spectral estimates are strongly biased toward harmonics, especially for high-pitched speech [3], and (2) the addition ofwhite noise to the Auto-Regressive (AR) process markedly
alters the spectral estimate [4]. These phenomena result
in deterioration of the perceived quality of re-synthesized
speech and can also cause speech recognition errors.
LP methods assume that the excitation signal conforms
to an Identically Independent Distributed (IID) Gaussian.
However, actual excitation signals exhibit non-stationary properties, especially in the case of a high fundamental frequency.
As a result, local peaks in the LP spectral envelope estimated from high-pitched speech are strongly biased towardharmonics. To correct this, we have proposed an AutoRegressive Hidden Markov Model (AR-HMM) and an accompanying parameter estimation method [5] in which the
HMM is introduced as a non-stationary excitation model.
We have also demonstrated that the proposed method canaccurately estimate the characteristics of both articulatory
systems and excitation signals from high-pitched speech.
Ozaki et al.[6] demonstrated that the recognition accuracy for a singing voice drastically deteriorates in comparison with that for normal speech, and that the deterioration
is caused by high-pitched sounds and prolonged sounds. In

0-7803-8874-7/05/$20.00 ©2005 IEEE

Previously, we proposed an AR-HMM that was obtained by
combining an AR process with an HMM introduced as anon-stationary excitation model. Figure 1 illustrates an example of the AR-HMM. The output probability distributionof each node in the excitation HMM is assumed to be a single Gaussian. The nodes in the ﬁgure are concatenated in aring state, so the state transitions occur in order. Therefore,
this type of AR-HMM can be used to represent periodically
excited signals. The AR-HMM can represent various typesof signals through appropriate design of the network topology. The number of nodes and the prediction order are determined according to the signal.

I - 237

Excitation HMM

AR(p)Fig. 1. Example of AR-HMM.

ICASSP 2005

➡

➡
3. ITERATIVE AR-HMM PARAMETERESTIMATION METHOD

4. If the likelihood has converged, the algorithm stops.
Otherwise, repeat the above processes for i ← i + 1
from step 2.

The AR-HMM parameters are the AR coefﬁcients and theparameters of the HMM. Previously, we presented an algorithm that iteratively estimates these parameters from asignal x(t), t = 0, · · · , T − 1 [5]. In the following, P denotes the prediction order of the AR process. Let a (i) =[a(i) (1), · · · , a (i) (P )]T represent the ith estimate of the AR
coefﬁcients. The ith estimate of the excitation signal e (i) (t), t =P, · · · , T − 1 is given by:(i)eP = xP − Ωa(i)

(1)

where

By repeating the above processes, the likelihood increases
almost monotonically in practical situations and converges
to the optimum or to a local optimum value.
The details of each step are as follows. In step 2, the AR
coefﬁcient vector can be obtained by
(i)

(i+1)

3.1 The Baum-Welch algorithm estimates the population(i+1)
2 (i+1)parameters µm , σm
, m ∈ S of each output(i+1)
distribution using e P .

(i)xt = [x(t) x(t + 1) · · · x(t + T − P − 1)] T ∈ RT −P ,3.2 The Viterbi algorithm estimates a state transition sequence s(t), t = P, P + 1, · · · , T − 1.

Ω = [xP −1 xP −2 · · · x 0 ] ∈ R (T −P )×P(i+1)

We allocate a unique number from S = {1, · · · , N } to eachnode of the excitation HMM to distinguish them from other
(i)
2 (i)
nodes, where N is the number of nodes. Let µn , σn , n ∈S represent the ith estimates of the output distribution population parameters in each node. Given a state-transition
sequence s(t) ∈ S, t = P, · · · , T −1, the population parameters of an excitation signal at time t are given by m (i) (t) =
(i)
2 (i)
µs(t) , v (i) (t) = σs(t) . Hence, the expectation vector of theexcitation signal vector is represented by:
(i)

(i)

ΣP = diag(v(i) (P ), v (i) (P + 1), · · · , v (i) (T − 1)) (3)The algorithm for parameter estimation consists of thefollowing processes.
1. The initial population parameters of the excitation sig(0)(0)
nal are prepared as mP = 0, ΣP = I. Repeat thefollowing processes from i = 0.
2. The AR coefﬁcients a(i+1) and the excitation signal(i+1)
eP
are estimated by maximizing the likelihood given
(i+1)
(i)
(i)
by L(eP ; mP , ΣP ).
(i+1)

and the diagonal co3.3 The expectation vector mP(i+1)
variance matrix ΣPof the excitation signal vector
are estimated using (2) and (3).
4. TRANSFORMATION OF AR-HMM
PARAMETERS TO MFCC
The transformation of the AR coefﬁcient to an LPC cepstrum can be efﬁciently computed using a simple recursive
formula.
n−1c(n) = −a(n) −(2)

Based on the assumption that the samples of the excitation
signal at different instants are mutually independent, the covariance matrix of the excitation signal vector is deﬁned asa diagonal matrix given by:

(i+1)

(i)

is derived from (1).
The excitation signal vector e P
In step 3, the population parameters of the excitation
signal vector are estimated according to the following processes.

eP = [e(i) (P ) e(i) (P + 1) · · · e(i) (T − 1)]T ∈ RT −P ,mP = [m(i) (P ) m(i) (P + 1) · · · m(i) (T − 1)]T(i)a(i+1) = [ΩT (ΣP )−1 Ω]−1 ΩT (ΣP )−1 (xP − mP ). (4), Σpof the ex3. The population parameters mp
citation signal vector are estimated by maximizing the
(i+1)
(i+1)
(i+1)
).
likelihood given by L(e P ; mP , Σp1(n − i)a(i)c(n − i).
n i=1

(5)

Also, we can obtain an LPC Mel-Cepstrum converting thefrequency axis to a Mel-frequency axis in a simple recursiveway.
Almost all recent Large-Vocabulary Continuous Speech
Recognition (LVCSR) decoders have adopted the MFCC asa speech feature, so it would be useful if the AR-HMMbased features could be directly recognized by the MFCCbased decoders. This requires transforming the AR-HMM
parameters into corresponding features of the MFCC. In order to do that, we processed the AR coefﬁcients according
to the following steps. The ﬁrst step is an evaluation of thelogarithmic spectral amplitude,


P





u(n) = − log 1 −(6)
a(i) exp(−i2πn/N ) ,
i=1

which corresponds to the logarithmic amplitude of the FFT
in a typical MFCC calculation. As a second step, Mel-ﬁlter
banks are constructed by summing the logarithmic spectral
amplitudes, weighted by triangle windows. Finally, we canobtain the AR-HMM-based MFCC by calculating the DCT
for the Mel-ﬁlter bank outputs.

I - 238

➡

➡
Table 2. Number of Nodes Selected
Song No.
3
4
7 11 21No.of Nodes 10 10 10 10 14Song No.
34 37 41 44 55
No.of Nodes 11 11 10 12 10

Table 1. Fundamental frequencies [Hz]Song No.
3
4
7
11
21
Avg.
315.3 279.4 416.9 261.5 409.5
Std.Dev. 84.38 89.69 76.54 54.99 72.04
Song No.
27
34
37
41
44
Avg.
275.1 342.2 305.1 234.1 225.0
Std.Dev. 60.01 99.26 48.39 44.80 61.12
Song No.
55
74
Avg.
365.5 216.3
Std.Dev. 105.6 30.21

27
10
74
10

Table 3. Acoustic scores evaluated by forced alignment
Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC

5. EXPERIMENTS
5.1. Popular Music Database
For our experiments, we used 12 Japanese songs of the popularmusic database “RWC Music Database: Popular Music”(RWC-MDB-P-2001 No. 3, 4, 7, 11, 21, 27, 34, 37, 41, 44,
55, 74) [7]. Instead of using the original audio signals after mixdown, we used the “Vocal-only Version” that records
only the vocal part. The singers of those 12 songs consistedof 5 females and 7 males.
The audio signals of the 12 songs were manually segmented into several vocal phrases of time interval ranging
from 2s to 6s. A total of 580 phrases were generated. Table
1 presents the average and standard deviation of fundamental frequencies of each song.
5.2. Acoustical Assessment of AR-HMM-Based Feature
In this section, we describe an acoustic assessment we performed of the AR-HMM-based MFCC. First, we extracted
the AR-HMM parameters from all the vocal phrases in all
12 songs. The analysis frame size was set to 25ms. The
frame shift was set to 10ms. The prediction order was set
to 16, and the number of nodes in the excitation HMM was
set to a range of 10 to 14. The evaluated AR coefﬁcientswere transformed into MFCC-compatible features using themethod described in Section 4. In addition, a conventional
MFCC was evaluated for comparison.
The number of nodes to be contained in the excitation
HMM for each singer was determined as follows. First,
we evaluated the forced-alignment acoustic scores for the
AR-HMM-based MFCC. The number of nodes was thendetermined by selecting the highest acoustic score. For the
acoustic score evaluations, we adopted a Phonetically Tied
Mixture (PTM) model, which was trained by using conventional MFCCs extracted from the continuous speech corpus of Japanese Newspaper Article Sentences (JNAS) [8].
Trained PTM model was thus completely open for singing
voices. The forced-alignment acoustic scores were evaluated using Julius, an LVCSR decoder [9].
Table 2 presents the ﬁnal node counts for the 12 songs.
Table 3 shows the evaluated acoustic scores for the selected
AR-HMM-based MFCC, as well as the acoustic scores forthe conventional MFCC. The acoustic scores are normalized by the number of frames. All of the acoustic scores3-24.47-27.0027-24.33-26.5555-25.39-28.044-24.77-27.3734-25.16-26.9474-23.98-26.687-24.97-27.2937-24.69-26.90Avg.
-24.85-27.12

11-24.41-26.3541-25.55-27.6621-26.47-28.5144-23.97-26.19for the AR-HMM-based MFCC were better than those of
conventional MFCC.
5.3. Recognition Results
Each vocal phrase was processed by Julius, the two-pass
LVCSR decoder [9]. A language model and a dictionary
were prepared for each song, which were generated fromthe lyrics using Chasen, a Japanese morphological analysis
system [10]. Each coefﬁcient of back-off smoothing was
set to an extremely small value. The acoustic models usedwere the same as those used for the acoustic assessment. A
weighting coefﬁcient for the language model and a word insertion penalty were optimized for each feature of the ARHMM-based MFCC and the conventional MFCC. Correct
word and error rates were evaluated from the recognition
results from the ﬁrst pass. The error rate was evaluated bysumming substitutions, deletions and insertions. Tables 4
and 5 present the correct word and error rates, respectively.
These results show that when compared with those from theconventional MFCC, the AR-HMM-based feature brought
improvements of 3.63% and 1.85% in correct word and error rates, respectively.
5.4. Model Adaptation
The recognition experiment described above was conductedunder the condition that the AR-HMM-based MFCC wasmismatched to the acoustic model. In this section, we prepared two adapted acoustic models. One model was adapted
to half the amount of the AR-HMM-based MFCCs, whichwere obtained from the vocal phrases of even numbers. The
other model was adapted to half the amount of the conventional MFCCs. Each Gaussian component in each acoustic model is transformed using a linear transformation estimated by means of Maximum Likelihood Linear Regression(MLLR). We iterated this adaptation process eight times.

I - 239

➡

➠Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC

Table 4. Correct word rate [%]34
7
1179.52 70.00 55.39 76.14
74.10 65.45 52.45 73.30
27
34
37
41
75.29 77.39 86.88 54.55
74.12 65.22 80.00 70.91
55
74
Avg.
42.22 92.53 70.04
51.85 61.49 66.41

Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC
Song No.
ARHMM
MFCC

Table 5. Error rate [%]3471127.71 31.82 62.75 37.50
33.73 36.36 62.75 32.39
27
34
37
41
35.29 30.43 17.50 49.09
33.53 46.09 21.25 30.91
55
74
Avg.
76.30 14.37 38.62
60.00 44.25 40.47

Table 6. Correct word rate [%] (using adapted models)Song No.
3
4
7
11
21
ARHMM 93.37 85.45 90.20 78.41 76.50
MFCC
91.57 75.45 86.27 80.11 48.85
Song No.
27
34
37
41
44
ARHMM 79.41 86.96 86.88 85.45 96.23
MFCC
90.59 79.13 82.50 81.82 92.89
Song No.
55
74
Avg.
ARHMM 74.07 85.63 84.88
MFCC
76.30 64.37 79.15

21
45.62
36.41
44
84.94
91.63

Table 7. Error rate [%] (using adapted models)Song No.
3
4
7
11
21
ARHMM 8.43 15.45 14.71 31.25 29.03
MFCC
11.45 24.55 19.61 27.84 56.68
Song No.
27
34
37
41
44
ARHMM 35.29 26.09 15.00 17.27 7.11
MFCC
10.00 31.30 18.12 18.18 11.30
Song No.
55
74
Avg.
ARHMM 40.74 18.97 21.61
MFCC
31.85 40.23 25.09

21
63.13
71.43
44
17.57
12.97

quencies,” Electronics and Communications in Japan,
Vol.53-A, No.1, pp.36-43, January 1970.
[2] B.S.Atal and S.L.Hanauer, “Speech analysis and synthesis by linear prediction of the speech wave,”
J. Acoust. Soc. Am., Vol.50, pp.637-644, 1971.

Tables 6 and 7 represent the results that the eighth adapted
acoustic models recognized the corresponding features, respectively. Comparing these results with those from the theconventional MFCC, the AR-HMM-based feature brought
improvements of 5.73% and 3.48% in the correct word andthe error rates, respectively. In comparison with the conventional MFCC recognized by acoustic models without adaptation, the AR-HMM-based MFCC with the model adaptation brought improvements of 18.47% and 18.86% in thecorrect word and the error rates, respectively.

[3] J.Makhoul, “Linear Prediction: A Tutorial Review,” inProc.of IEEE, Vol.63, No.4, pp.561-580, April 1975.
[4] S.M.Kay, “The Effects of Noise on the Autoregressive
Spectral Estimator,” IEEE ASSP-27, No.5, pp.478485, Oct. 1979.

6. CONCLUSIONS[5] A.Sasou, M.Goto, S.Hayamizu, K.Tanaka, “Comparison of Auto-Regressive, Non-Stationary Excited Signal Parameter Estimation Methods,”
Proc. of IEEE MLSP, Sep. 2004.

In this paper, we applied AR-HMM to extraction of a feature from singing voices. The effectiveness of the AR-HMMbased feature was conﬁrmed from acoustic assessments andsinging-voice recognition experiments.
In future studies, moreover, to achieve higher recognition accuracy for a singing voice, the inﬂuence of prolonged
sounds should be taken into account. We also need to consider formant variations, particular with singing voices. For
instance, the singing-formant appears in the frequency bandfrom 2.5kHz to 3kHz in singing voices. Another example
is that the ﬁrst formant frequency tends to shift to the fundamental frequency when that frequency is higher than theﬁrst formant frequency.

[6] H.Ozeki, T.Kamata, M.Goto, S.Hayamizu, “The inﬂuence of vocal pitch on lyrics recognition of sung
melodies,” Proc. of Acoust. soc. Japan, Vol.1, pp.637638, Sep. 2003(in Japanese).
[7] M.Goto, “Development of the RWC Music Database”,
Proc. of ICA 2004, pp.I-553-556, April 2004.
[8] K.Itoh et al., “JNAS: Japanese speech corpus for large
vocabulary continuous speech recognition research,”
J. Acoust. soc. Japan (E), Vol.20, No.3, pp.199-206,
March, 1999.
[9] http://julius.sourceforge.jp/en/julius.html

7. REFERENCES[1] F.Itakura and S.Saito, “A statistical method for estimation of speech spectral density and formant fre-[10] http://chasen.naist.jp/hiki/ChaSen/

I - 240