1-Q-25(e)Towards a Statistical Audio Signal Processing Framework
Based on the I-Divergence∗○ Hirokazu Kameoka
NTT Communication Science Laboratories

1Introduction

For many applications in statistical signal processing, a statistical model in which a short-time signal is
assumed to be a set of samples drawn from a zero-mean
stationary Gaussian process is very often employed.
In the frequency domain, this assumption amounts toconsidering that each frequency component of the signal is generated according to a zero-mean circular complex normal distribution with a diﬀerent variance. Thevariance of the distribution in this context is usuallycalled the power spectral density (PSD). The maximum likelihood estimation problem under this statistical model involves determining the PSD estimate froman observed signal, which is shown to be equivalent tominimizing a divergence measure called the ItakuraSaito (IS) divergence between the sample power spectrum and the PSD of the assumed stochastic process[1]. While the solution is obvious when no modelingconstraints are imposed on the PSD, it is generally
necessary to develop an appropriate optimization algorithm according to this criterion when the PSD is assumed to have a certain structure that can be describedusing a small number of parameters (for example, using
an all-pole model or non-negative matrix factorization
model). However, for some classes of parametric models the cost function is sometimes numerically diﬃcult
to optimize, due to the highly non-convex nature of thedivergence function. For example, as reported in [2],it has been found in practice to be prone to numericalinstability and local minima during optimization whenapplied as a goodness-of-ﬁt criterion for non-negative
matrix factorization (NMF).
In this paper, we focus on another divergence measure between two non-negative functions, called the Idivergence [3], which has several remarkable features.
Firstly, Csiszár showed in [3] that under non-negativity
constraints, the only discrepancy measure consistent
with certain fundamental axioms such as locality, regularity and composition-consistency is the I-divergence.
Secondly, according to the results obtained in NMFbased single channel source separation tasks undermany diﬀerent model-ﬁtting criteria [4], the use of theI-divergence has been found to provide the best performance. This should indicate that the stochastic process assumption underlying the I-divergence is describing the actual statistics of audio signals well. Thirdly,
it is mathematically convenient to derive optimization
algorithms for particular classes of parametric spec∗

trum models such as those introduced in [5, 6]. Here,one intriguing question arises: What kind of stochastic
process are we implicitly assuming when we choose to
use the I-divergence as a spectral distortion measure?
The objective of this paper is to answer this question.

2Review of Itakura-Saito Divergence

In this section, we brieﬂy review the way in which theIS divergence is derived from the stationary Gaussianrandom process assumption. Let x = (x1 , · · · , xI )T ∈RI be a real vector of a discrete-time signal, that isassumed to have been drawn from a zero-mean Gaussian random process x ∼ N (0, Σ). The Fourier transform of x, given by z = F x ∈ CI , follows a zeromean multivariate complex normal distribution withcovariance matrix F ΣF H , where F ∈ CI×I is the discrete Fourier transform matrix. If we assume stationarity (and circularity), the covariance matrix Σ belongs to the class of nonnegative deﬁnite symmetric
Toeplitz circulant I × I dimensional matrices. Σ isthen shown to be exactly diagonalized by F so that
we obtain F ΣF H = diag(λ1 , · · · , λI ) where λ1 , · · · , λI
are the eigenvalues of Σ, corresponding to the power
spectral densities (PSDs). This indicates that each element of z, zi (1 ≤ i ≤ I), independently follows azero-mean complex normal distribution with variance
λi : zi ∼ NC (0, λi ). Now, suppose that we are given
a Fourier component, z. We consider the maximum
likelihood estimation of λ on the basis of the generative model z ∼ NC (0, λ). By diﬀerentiating the loglikelihood LN (λ) = − log πλ − |z|2 /λ with respect to λand setting the result at zero, we determine the maximum likelihood estimator λ = |z|2 and hence we obtain
LN (|z|2 ) ≥ LN (λ). By subtracting the right-hand sideof this inequality from the left-hand side, we obtain a
non-negative measure:
LN (|z|2 ) − LN (λ) =|z|2|z|2− log− 1 ≥ 0,λλ(1)which is equal to the IS divergence [1]. This quantity
is shown to reach 0 only when λ = |z|2 .

3Complex Poisson Distribution

We propose a probability density function of a complex number z, called the complex Poisson distribution, mention some of its properties, and then showthat this distribution leads to the I-divergence in thesame way that we derived the IS divergence from thecomplex normal distribution in the previous section.

I ダイバージェンスに基づく統計的音響信号処理の枠組に向けて, 亀岡弘和 (NTT CS 研)

日本音響学会講演論文集- 813 -2011年3月

201010100-10-20-20-10010Imaginary part

20Imaginary part

Imaginary part200-10-20-2020-10010200-10-20-20-10Real part

Real part

01020Real part

Proof: This can be easily conﬁrmed by considering only the terms involving λ in the log-likelihood,LP (λ) = −λ + |z|p log λ, which is maximized when
λ = |c|p . Thus, LP (|z|p ) ≥ LP (λ). Subtracting theright-hand side of this inequality from the left-handside gives a non-negative measure

Fig. 1Illustration of how the density function changes according to the parameter settings: (left) (λ, p) = (5, 1), (middle)(λ, p) = (10, 1), and (right) (λ, p) = (102 , 2).

Deﬁnition 1 (Complex Poisson Distribution). The
complex Poisson distribution is a density function ofa circular complex random variable z deﬁned over thesupport z ∈ D = {z ∈ C | |z|p ∈ N} such thatpe−λ |z|p−2 λ|z|,2π(|z|p )!
pfz (z; λ, p) =(2)LP (|z|p ) − LP (λ) = |z|p log

which is equal to the I-divergence.

4

Construction of Stationary Process

Given a set consisting of Fourier components z =(z1 , . . . , zI )T , let us assume that each component
has been generated independently according to zi ∼cPois(λi , p). The probability of z being generated iswhere λ ∈ R≥0 and p ∈ R>0 are the parameters characterizing its distribution. We use the notationfz (z; λ, p) =I|z |∏pe−λi |zi |p−2 λ iii=1z ∼ cPois(λ, p),(3)

to indicate that a complex-valued random variable z
follows a complex Poisson distribution.
Fig. 1 is an illustration of the complex Poisson distributions with diﬀerent parameter settings. In thefollowing we show several important properties related
to the proposed circular distribution.
Property 1. The integral of fz over the support D is∫fz (z; λ, p)dz = 1.
(4)
D

Property 2. If z follows a complex Poisson distribution z ∼ cPois(λ, p), then k = |z|p ∈ N follows aPoisson distribution with mean λ:k∼k −λλ ek!

.

(5)Property 3. If z follows a complex Poisson distribution such that z ∼ cPois(λ, p), the qth-order moments
deﬁned by βn;m := E[z n z ∗m ] [7], where n and m arenatural numbers such that q = n + m, are given by{ ′µq/p (n = m)βn;m =,(6)0(n ̸= m)where µ′ξ denotes the fractional moment of order ξ > 0of the Poisson distribution with mean λ.
Example 1. It follows from Property 3 that the mean
E[z] and the variance V[z] of z ∼ cPois(λ, p) are{ 2λ + λ (p = 1)
E[z] = 0, V[z] =.
(7)λ(p = 2)
Property 4. The maximum likelihood estimation of λunder the generative model z ∼ cPois(λ, p) is equivalent to the problem of minimizing the I-divergence between |z|p and λ.

日本音響学会講演論文集|z|p− (|z|p − λ) ≥ 0, (8)λ2π(|zi |p )!

p,(9)where λ = {λi }1≤i≤I . We can easily show from whatkind of probability distribution the time-domain signal, x = F H z, is supposed to have been generated under this assumption. Since the (inverse) Fourier transform is a unitary transform, we know that | det F | = 1.
The density function of x can thus be described interms of fz such that fx (x; λ, p) = fz (F x; λ, p).
It is important to note that when p = 2, λ corresponds to the PSD of the assumed stochastic process(see Example 1), in which case the maximum likelihood estimation of λ can be understood as the problem of ﬁtting the PSD to the sample power spectrum{|zi |2 }1≤i≤I under the I-divergence criterion.

5Conclusion

In this paper, we proposed a new probability density
function of a circular complex random variable, mentioned some of its properties, and showed that this distribution can be used to construct a stochastic processthat supports the use of the I-divergence as a wellfounded spectral distortion measure. By using thisstochastic process, we should be able to build a welldeﬁned audio signal processing framework, allowing forthe introduction of all kinds of spectrum models which
behave well with respect to the I-divergence criterion.

Acknowledgement
The author would like to thank Dr. Naonori Ueda
(NTT) for his fruitful discussions.

References[1] Itakura, Ph.D. thesis, Nagoya University, 1972.
[2] Bertin et al., Proc. ICASSP’09, pp. 840–843, 2009.
[3] Csiszár, Ann. Stat., 19(4), pp. 2032–2066, 1991.
[4] FitzGerald et al., Proc. ISSC’09, 2009.
[5] Kameoka, Ph.D. thesis, University of Tokyo, 2007.
[6] Kameoka, IEICE Tech. Rep., SP2010-74, pp. 29–34, 2010.
[7] Amblard et al., Signal Process., 53(1), pp. 1–13, 1996.

- 814 -2011年3月