Mel-Generalized Cepstral Representation of Speech—A Uniﬁed Approach to Speech Spectral Estimation

Keiichi Tokuda
Nagoya Institute of Technology
Carnegie Mellon University

Tamkang UniversityMarch 13, 2002

1

Conventional Speech Spectral Estimation• Linear prediction (LPC)• Cepstral analysis• Subband ﬁlter bank

Autoregressive (AR) model
Exponential (EX) modelNonparametricVariations• Model• Analysis window⇒ Pole-zero (ARMA) model⇒ Adaptive analysis(sample by sample basis)• Auditory characteristics ⇒ Warped LPC, PLP, etc.
◦ Auditory frequency scales (mel, Bark)◦ Loudness scales (log, sone)2

Structure of This Talk

1.

Conventional cepstral analysis

2.

Introduction of generalized logarithmic function⇒

3.

Introduction of auditory frequency scale⇒

4.

Generalized cepstral analysis

Mel-generalized cepstral analysis

Applications to speech recognition and coding

3

History of Cepstral Analysis• B.P. Bogert, M.J.R. Healy, J.W. Tukey (1963)
Analysis of seismic signals— decomposition into direct wave and echo
⇒ Cepstrum, Quefrency, Lifter• A.M. Noll (1964, 1967)
Pitch extraction based on cepstrum• A.V. Oppenheim (1966, 1968)
Homomorphic deconvolution— decomposition into source and vocal tract function⇒ Complex cepstrum

4

Deﬁnition of Cepstrum

Fourier transform of signal s(n)S(ejω ) = F [ s(n) ]Cepstrum−1jω
2C(m) = F
log |S(e )|(Bogert et al., Noll)



−1
jω
log |S(e )|C(m) = F(Oppenheim)

5Complex Cepstrum

z-transform of signal s(n)S(z) = Z [ s(n) ]Complex cepstrumc(m) = Z −1 [ log S(z) ]



−1jω
log S(e )= F−1jωjω
= F
log |S(e )| + j arg S(e )6Cepstrum and Complex Cepstrum

log |S(ejω )| = F [ C(m) ] = Re [ F [ c(m) ] ]⇓When it is minimum phase (all polesand zeros are located in the unit circle)⎧⎪⎪⎨ 0,m<0c(m) = C(m), m = 0⎪⎪⎩2C(m), m > 0

7

Homomorphic Deconvolution

Pulse traine(n)hPPPPPP hLinear
time-invariantsystemhs(n) = e(n) ∗ h(n)-Speechh(n)White noises(n) = h(n) ∗ e(n)↓FS(ejω ) = H(ejω )E(ejω )↓log | · |log |S(ejω )| = log |H(ejω )| + log |E(ejω )|↓F −1C(m) = Ch(m) + Ce(m)800

Amplitude (dB)

s(n)

100

100200Time (100 μ s)
(a)

C(m)

50040
200

1234
Frequency (kHz)(c)

501234Frequency (kHz)(e)

550
Amplitude (dB)

Amplitude (dB)

60

050
100
Quefrency (100μ s)
(b)

10080
60
40
20080

01234
Frequency (kHz)
(d)

50-50

9Spectrum of Periodic Signaljω
log⏐Ew(e )⏐jω⏐W(e )⏐w(n)
e(n)

0

Np

2π /NpFrequencyπjω

h(n)

s(n)

jω
log⏐Sw(e )⏐e(n)⏐H(e )⏐0Frequencyπ10

Problems of Homomorphic Processing(Cepstral Analysis)

Linear smoothing of log spectrum• aﬀected by ﬁne structure of FFT spectrum• results in a large bias and variance

Voiced speech (periodic)• Envelope of peaks of spectral ﬁne structure⇒ Improved cepstral analysis , PSE: Biased

11

Cost Function

P (ω):IN (ω):

Estimate of Power Spectrum
Periodogram1 π IN (ω)
IN (ω)
E=− log−12π −π P (ω)
P (ω)

dω ⇒ minx: Gaussian Process ⇒ Maximizing p(x|c)• Unbiased estimation of log spectrum• equivalent to one used in LPC• Minimization of energy of inverse ﬁlter output

12

Analysis of Natural Speech
80
Log magnitude (dB)

Log magnitude (dB)

80
60
40
200-200

1234
Frequency (kHz)5(a) Unbiased cepstral analysis

6040200-2001234
Frequency (kHz)5(b) Linear prediction

13Generalized Cepstrum

Complex Cepstrumc(m) = Z −1 [ log S(z) ]log S(z) = Z [ c(m) ]⇓Generalized Cepstrumcγ (m) = Z −1 [ sγ (S(z)) ]sγ (S(z)) = Z [ cγ (m) ]14

Generalized logarithmic function⎧⎨ (w γ − 1)/γ, 0 < |γ| ≤ 1sγ (w) =⎩ log w,γ=0sγ ( x )γ =10 <γ <1γ = 0 (log x )101−1 < γ < 0γ = −1x15

Spectral Model

Generalized Cepstrum: cγ (m)⎛⎝H(z) = s−1γM⎞cγ (m) z −m⎠m=0⎧ ⎛⎞1/γ⎪M⎪⎪⎪−m⎠⎝1 + γ⎪c(m)z, 0 < |γ| ≤ 1⎪γ⎨m=0=⎪M⎪⎪⎪−m,⎪expc(m)zγ=0⎪γ⎩m=0Inverse function of Generalized logarithm⎧⎨ (1 + γw)1/γ , 0 < |γ| ≤ 1−1sγ (w) =⎩ exp w,
γ=016

Cost Function

1 π IN (ω)
I (ω)
E=− log N−12π −π P (ω)
P (ω)

dω ⇒ minEstimate of Power Spectrum
P (ω) = |H(ejω )|2 = σ 2|D(ejω )|2Interpretation in time-domain
2ε = E e (n) ⇒ minx(n)e(n)1/D(z)-17Advantage−1 ≤ γ ≤ 0:
• Convex function ⇒ Global solutioncan easily be obtained• The obtained system H(z) is minimum phase, e.g., stable• γ = −1 ⇒ Linear Prediction
1H(z) =1−Mcγ (m)z −mm=0• γ = 0 ⇒ Cepstrum
MH(z) = exp

cγ (m)z −m

m=018

Prediction GainPredictor:• D(z) is minimum phase• Gain of D(z) is one

ε = E e2(n)2E x (n)G= 2E e (n)a(k)z −kQ(z) =k=1-⇒ Prediction Gain:⇒x(n)Cost Function:∞x(n)

1/D(z)⇓t-e(n)
-e(n)
+−

6Q(z)

19

Analysis of synthetic signal

True
γ =-1-3/4-1/2-1/4
010dB
02.55Frequency(Hz)Prediction gain(dB)

Log magnitude(Generalized Cepstral Analysis)

8M=876-1(LPC)-0.5
γ0(UELS)(a) Example 120

Analysis of synthetic signal

True
γ =-1-3/4-1/2-1/4
010dB
0
2.5
5Frequency(Hz)Prediction gain(dB)

Log magnitude(Generalized Cepstral Analysis)

5M=843-1(LPC)-0.5
γ0(UELS)(c) Example 321

Analysis of synthetic signal

True
γ =-1-3/4-1/2-1/4010dB
02.55Frequency(Hz)Prediction gain(dB)

Log magnitude(Generalized Cepstral Analysis)

M=876-1(LPC)-0.5
γ0(UELS)(b) Example 222

Analysis of natural speech(Generalized Cepstral Analysis) /e/γ = -160

γ = -1/2

γ = -1/3

γ = 040200-2001 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5Frequency(kHz)Frequency(kHz)Frequency(kHz)Frequency(kHz)Prediction gain(dB)

Magnitude(dB)8016M=1515-1(LPC)-0.5
γ0(UELS)(a) male /e/23

Analysis of natural speech(Generalized Cepstral Analysis) /N/
γ = -160

γ = -1/2

γ = -1/3

γ = 040200-200

1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5Frequency(kHz)Frequency(kHz)Frequency(kHz)Frequency(kHz)Prediction gain(dB)

Magnitude(dB)80

M=152827-1(LPC)-0.5
γ0(UELS)(b) male /N/24

Structure of synthesis ﬁlter H(z) (γ = −1/n)
1stinput ZZσZ-1C(z)-3nd. . .. . .. . .

1C(z)

−−−−

1H(z) = σD(z) = σC(z)⎛C(z) = ⎝1 + γMn-th-

1C(z)output-n⎞cγ (m) z −m⎠m=025

Structure of synthesis ﬁlter H(z) (γ = 0)
—LMA ﬁlter
Output-+nInputn s-+
7 COCSoS - C S -s

F (z)AL,1 JJs

F (z)AL,2 JJss

F (z)

Js

7 COCSoS  C SF (z)AL,3 J

AL,4 JJssL

AL,l {F (z)}l1+D(z) = exp F (z)-l=1LRL(F (z)) =AL,l {−F (z)}l1+l=1MF (z) =cγ (m) z −mm=126

Introduction of auditory frequency scale

πz −1 − α−1zα = Ψ(z) =1 − αz −1
Phase Characteristics can beused for Frequency
Transformation:2 ) sin ω(1−αω̃ = tan−1(1 + α2) cos ω − 2αwhere Ψ(ejω ) = e−jω̃Warped frequency ω∼ (rad)First-order all-pass function:mel scale

π /210kHz sampling
α = 0.3500π /2Frequency ω (rad)π27Mel-Generalized Cepstral Analysis

Mel-generalized cepstrum: cα,γ (m)⎛⎞M−1−m⎠
cα,γ (m) zα
H(z) = sγ ⎝m=0⎧ ⎛⎞1/γ⎪M⎪⎪⎪−m⎠⎝1 + γ⎪c(m)z, 0 < |γ| ≤ 1⎪α,γα⎨m=0=⎪M
⎪
⎪
⎪
−m,
⎪exp
c(m)
z
γ=0
⎪
α,γα
⎩
m=0



z −1 − α−1zα =−1

1 − αz28•(α, γ) = (0, 0) ⇒ Cepstral model:MH(z) = exp

cα,γ (m)z −mm=0•(α, γ) = (0, −1) ⇒ AR model:
1H(z) =1−Mcα,γ (m)z −mm=0•(α, γ) = (0.35, 0) ⇒ Mel-cepstral model:MH(z) = exp−mcα,γ (m) zαm=0•(α, γ) = (0.47, −1) ⇒ Warped AR model:
1H(z) =1−

M−mcα,γ (m) zα

m=029

A Uniﬁed Approach to Speech Spectral Estimation|α| < 1, −1 ≤ γ ≤ 0''α=0Generalized cepstral analysis$$

Mel-generalized cepstralanalysis
γ = −1'

Linear prediction

Warped Linear Prediction&%'Unbiased Cepstral analysis

γ=0&

$

Mel-cepstral analysis&&$%%%

30

Mel-generalized analysis of natural speech /N/ M = 12

Magnitude(dB)

Magnitude(dB)

Magnitude(dB)

α=0

α = 0.35

α = 0.4740
γ =00-4040
γ = − 0.50-4040γ = −10-4002.5Frequency(kHz)

5 0

2.5
Frequency(kHz)

5 0

2.5
Frequency(kHz)531

Example
α=0na

γ = −1γ = −1/3

γ=0Frequency(kHz) Frequency(kHz) Frequency(kHz)0012345b udew
400(a) Waveforma(ms)40(dB)(α, γ , M) = (0, -1, 12)

LP40(dB)(α, γ , M) = (0, -1/3, 12)

GCEP012345012345N
200(α, γ , M) = (0, 0, 12)
UELS40(dB)(b) Spectral estimates ( α = 0)32

Exampleα = 0.35na

γ = −1γ = −1/3

γ=0Frequency(kHz) Frequency(kHz) Frequency(kHz)0012345012345b udew
400(a) Waveforma(ms)(α, γ , M) = (0.35, -1, 12)

WLP40(dB) (α, γ , M) = (0.35, -1/3, 12)

MGCEP40(dB)
012345N200

MCEP40(dB) (α, γ , M) = (0.35, 0, 12)(b) Spectral estimates ( α = 0.35)33Structure of synthesis ﬁlter H(z) (γ = −1/n)
1stinput ZZ
σZ


-



1C(z)

-

2nd. . .. . .. . .

1C(z)−−−−n-thoutput

1C(z)

-

-

Structure of H(z)

H(z) = σD(z) = σ

C(z) =1C(z)n6cα,γ (m) zα−m−+l@I@@@@A A@@αM

1+γOutput@A Ab(1)γ(1 − α2 )
HH?

+l

@@@@?
t- l tz −1+A Ab(2)@@@A A

@α@Input?
-+lz −1 -+l t z −1−6m=0A A

−+l@
I@@

α@@?
t- l tz −1+A A

b(3)?

+l

Structure of C(z) (M = 3)34

Structure of synthesis ﬁlter H(z) (γ = 0)
—MLSA ﬁlter

D(z) = exp F (z)MF (z) =RL(F (z))−mcα,γ (m) zαm=0• suﬃcient accuracy: maximum spectral error 0.24dB• O(8M ) multiply-add operations a sample• guaranteed stability• M multiply-add operations for ﬁlter coeﬃcients calculation

35

Structure ofMLSA ﬁlter

Output-+nInputn s-+ COCSoS7 - C S -s

F (z)sF (z)AL,1 J

AL,2 J

JRL(F (z))

QInputQs

exp F (z) = D(z), L = 4- +m


@@

Jα
Jz −1Js@@

1 − α2

AL,4 JJs7 COCSoS  C SF (z)AL,3 JJss

F (z)

-@@?
m r
-+

Jb(1) J@z −1@I@@JαJ@- +m@?
r -+m rJb(2) Jz@@@I@JαJ@@@@?
r -+m r−1@

@

z −1

Jb(3) J?
-+m?
-+mOutput-

Basic ﬁlter F (z), M = 336

The choice of α, γ for speech analysis/synthesis

Analysis/synthesis system with ﬁxed α and γ• speech quality change with γ
γ → −1 Clearγ→0Smooth• When γ = 0, speech quality with (α, M ) = (0.35, 15) isalmost equivalent to that with (α, M ) = (0, 30).
• When the analysis order is high enough, the diﬀerence
becomes small.

37

Feature of Uniﬁed Approach• Linear prediction analysis, Cepstral analysis are the special cases.
• Mathematically well-deﬁned• Physical interpretation⇒ Minimization of energy of inverse ﬁlter output⇒ x is Gaussian ⇒ Minimization of p(x|c) (ML estimation)• Global solution, stability of the system function• Synthesis ﬁlter for direct synthesis from the estimated coeﬃcients⇒ LMA/MLSA/GMSLA ﬁlter• Extension to adaptive analysis (sample by sample basis)• Parameter transformation for speech recognition38

Word Recognition based on HMM

Spectral Analysis:
1. (α1, γ1, M1) = (0, −1, 12) ⇒ Linear Prediction
2. (α1, γ1, M1) = (0.35, −1/3, 12) ⇒ Mel-generalized cepstralanalysis
3. (α1, γ1, M1) = (0.35, 0, 12) ⇒ Mel-cepstral analysis

Output vector of HMM:
(α2, γ2, M2) = (0.35, 0, 12)
Mel-cepstral coeﬃcientsand Δ (dynamic coeﬃcients)⎛⎝
H(z) = s−1
γ1

M1
m=0⎞

⎛−m ⎠ = s−1 ⎝cα1,γ1 (m) zα
γ21∞⎞
−m⎠
cα2,γ2 (m) zα2

m=039

Recognition Accuracy(Continuous HMM, 33 phoneme models, 2618 words)

Linear Prediction(α 1, γ 1, M 1)=(0, -1, 12)
Mel-Generalized
Cepstral Analysis(α 1, γ 1, M 1)=(0.35, -1/3, 12)
Mel-CepstralAnalysis(α 1, γ 1, M 1)=(0.35, 0, 12)

77.5%

80.7%

79.6%
7580
Recognition accuracy (%)
40

Application to 16kb/s wideband CELP coder
Input
Quantizationof MGC Coef.

MGCAnalysis
MGC coef.

encoder

CELP
ExcitationGenerator

Synthesis
Filter
MGC coef.

Minimize
MSE

PerceptualWeighting
Filter

MGC coef.

decoder

CELP
Excitation
Generator

Synthesis
Filter

MGC coef.

OutputPostfilter

41

Speech quality as a function of α (γ = −1/2)
4.5

DMOS

4.0

3.5
averagefemale
male
3.0
0.0

0.1

0.2

0.3

0.4

0.5

α42

Subjective Evaluation
MNRU(dB) 162432

40

48

G.722
48kb/s
G.722
56kb/s
G.722
64kb/sConv.CELP
16kb/s
MGC-CELP
16kb/s123DMOSaveragefemale45
male43

Summary
A uniﬁed approach to speech spectral estimation• A uniﬁed approach toLinear predicton and Cepstral analysis• Introduction of auditory frequency scale• Eﬃcint representation of speech spectrum with anappropriate choice of α and γ• Application to speech anaylysis/synthesis, speechcoding, speech recognition

Future work:

Optimal α and γ(Phoneme/Speaker dependent?)

Speech Signal Processing Toolkit:
http://kt-lab.ics.nitech.ac.jp/~tokuda/SPTK/44