UNSUPERVISED MUSIC UNDERSTANDING
BASED ON NONPARAMETRIC BAYESIAN MODELS
Kazuyoshi Yoshii

Masataka Goto

National Institute of Advanced Industrial Science and Technology (AIST){k.yoshii, m.goto}@aist.go.jp
ABSTRACTThis paper presents a new research framework for unsupervised music understanding. Our goal is to recognize musical notes from polyphonic audio signals and simultaneously induce grammatical patterns from the recognized notes by integrating probabilistic acoustic
and language models. Given music audio signals, both models could
be jointly trained in a self-organizing manner without manually specifying the numbers of musical notes and grammatical patterns. In
this paper, we introduce our nonparametric Bayesian acoustic andlanguage models for multipitch analysis and chord progression analysis and discuss issues for integrating these models. We then provide
a novel overview of various acoustic and language models whose underlying concepts are useful for implementing the framework.
Index Terms— Unsupervised music understanding, Bayesiannonparametrics, statistical machine learning, acoustic and language
models, music transcription, grammar induction
1. INTRODUCTION
Music is one of the most sophisticated forms of audio signals. Evenmusically-untrained people can enjoy music and intuitively perceivethat musical notes are organized to form sequential and simultaneous structures. Even if they have not been taught labels like C major
and D minor, they intuitively know that only particular note combinations can sound harmonically (form chords). We assume that thismusical sense can be acquired by just listening to a large amountof music, i.e., that people are capable of unsupervised music understanding. This capability is the basis of the analytical music listeningthat makes it easy for us to distinguish individual musical notes infamiliar musical pieces. More speciﬁcally, humans intuitively grasp
typical structural patterns that commonly appear in musical pieces
they have listened to, and they use those patterns as grammaticalclues for distinguishing overlapped notes. The recognized notes arein turn used in the deeper discovery of structural patterns.
In the light of the above discussion we propose a novel machinelearning framework for unsupervised music understanding (Fig. 1).
We aim to recognize musical notes from polyphonic audio signals
and simultaneously induce structural patterns from the recognizednotes in an unsupervised manner. So far, converting continuous data
(audio signals) into discrete data (musical notes) has been actively
investigated, but pure audio-note modeling is insufﬁcient as “music”
signal processing because it does not consider grammatical patternsof musical notes (e.g. chords). We assume that what makes music“music” lies in the discrete world because most music can be denotedin symbolic representations (music scores). Inter-note modeling isthus indispensable for music understanding.
Our hierarchical framework is similar to the typical frameworkof automatic speech recognition (ASR), in which speech signals areThis study was supported in part by the JSPS KAKENHI 23700184.

978-1-4673-0046-9/12/$26.00 ©2012 IEEE

5353

Prior

p (S )

Grammatical patterns S
Grammarinduction

Language
model

Note progressions
Note combinations (chords)
Chord progressionsetc.

p( Z | S )Musical notes Z
Musictranscription

Acousticmodelp( X | Z )Music audio signals X

Fig. 1. A hierarchical Bayesian framework for unsupervised music
understanding based on integration of acoustic and language models.
transcribed into words (sequences of phonemes) by using an acoustic model that represents the spectral dynamics of speech signals anda language model that represents the connections between words.
Such audio-phoneme modeling and inter-phoneme modeling are performed under the condition that all ﬁxed vocabularies of linguistic
units (e.g., phonemes and words) are explicitly given as an oracle.
A key difference of “unsupervised” music understanding is that an
adequate number of discrete “musical units” (e.g., notes and chords)
should be inferred from continuous music audio signals without using any ground-truth vocabularies. This is analogous to language acquisition by infants, who are capable of discovering linguistic units
in continuous speech signals in an unsupervised manner.
A principled approach to unsupervised music understanding is toformulate probabilistic acoustic and language models independently
and then integrate them in a hierarchical Bayesian manner. A major
difﬁculty in the recognition of musical notes is that the number ofmusical notes contained in audio signals is unknown. Therefore, thecomplexity of the acoustic model should be appropriately adjusted.
In the induction of structural patterns, the complexity of the language
model should also be adjusted according to the structural regularityof recognized notes. Conventional Bayesian models, however, force
us to specify these complexities in advance even though the combinatorial search for optimal complexities is impractically expensive.
Nonparametric Bayesian models, on the other hand, are free fromthis problem because they can not only learn their parameters (howlikely each note or chord is to be used) but also efﬁciently adjust theirown effective complexities (how many notes and chords should beconsidered) [1]. Such “musical units” can be generated unboundedly
if they are needed to explain the given music data.
The remainder of this paper is organized as follows: Section 2
explains the probabilistic framework of unsupervised music understanding. Section 3 and Section 4 provide a novel overview of various acoustic and language models that could be used as componentsof the framework. Section 5 concludes the paper.

ICASSP 2012

3. PROBABILISTIC ACOUSTIC MODELSThe goal of music transcription to convert polyphonic music audio
signals into musical notes, i.e., discrete symbols that have absolute
pitches (C0, C#0, D0, · · · ) and relative durations (whole, half, quarter, · · · ). Conventional acoustic models have been designed only for
estimating fundamental frequencies (F0s) of musical sounds at every
frame [2, 4, 5] or for separating musical sounds [6–11]. Some models [5] take into account the temporal continuity of frame-level F0s
for detecting onsets and offsets of musical sounds. Note that the F0s
and durations of musical sounds take continuous values represented
by hertz and seconds. This means that it is implicitly assumed that at
a post-processing stage the F0s are discretized by the semitone andthe durations are quantized according to music tempo. This makes it
difﬁcult, however, to directly represent structural patterns (e.g., combinations and progressions) of discrete symbols (musical notes) in aprobabilistic and principled manner.
In the framework of unsupervised music understanding, we needto build a new type of probabilistic acoustic models that can accountfor an inﬁnite number of musical notes having inﬁnitely many kindsof discrete pitches and durations. To represent p(X|Z), i.e., how

5354

Prob. density Prob. density

We explain a mathematical formulation of the probabilistic framework for unsupervised music understanding. As shown in Fig. 1,
the three kinds of random variables, X, Z, and S respectively denote music audio signals or frequency spectra, musical notes, andstructural patterns of those notes such as chords (note combinations)and progressions. We propose a fully hierarchical Bayesian modelp(X, Z, S) = p(X|Z)p(Z|S)p(S), where p(X|Z) and p(Z|S)
are called acoustic and language models, respectively, and p(S) is aprior distribution over structural patterns. In this study p(Z|S)p(S)is regarded as a language model in a broad sense.
Our goal is to infer latent variables Z and S from observed data
X in an unsupervised manner. For Bayesian inference, we aim tocompute a posterior distribution over Z and S by using Bayes’ rule,i.e., p(S, Z|X) = p(X, Z, S)/p(X), where a marginal likelihood(a.k.a. evidence) is given by p(X) = p(X|Z)p(Z|S)p(S)dZdS.
Although p(X) is analytically intractable in general, we need to use
approximate inference methods such as the variational Bayes (VB)
and Markov-chain Monte Carlo (MCMC). If necessary, we can takethe maximum-a-posteriori (MAP) point estimates of Z and S fromthe posterior distribution p(S, Z|X). This data-driven framework
enables us to compare different implementations of acoustic and language models in terms of the uniﬁed criterion p(X). In addition, we
can adapt this framework to a semi-supervised setting in which thesize of a latent space is ﬁxed and/or the values of latent variables are
partially given as ground-truth deﬁnitions (musical knowledge).
One reason that we need nonparametric Bayesian models to implement p(X|Z) and p(Z|S)p(S) is that the space of Z and thatof S are inﬁnite in theory. Note that any additional knowledge (e.g.,
how many musical notes and chords are contained in observed dataX) is not available in the completely unsupervised setting. If an inﬁnite amount of X were available, inﬁnitely many kinds of musicalnotes and structural patterns would be needed to represent an inﬁnite variety of X. When the amount of X is ﬁnite, however, limited
but unknown numbers of notes and patterns need to be considered A
practically-important feature of nonparametric Bayesian models isthat they theoretically have inﬁnite complexity but actually instantiate only the numbers of musical notes and structural patterns necessary to represent X. This enables us to manage such inﬁnite models
on real computers having ﬁnite computational power.

Observed count

2. UNSUPERVISED MUSIC UNDERSTANDING

Component distribution corresponding to musical sound #1

Frequency

Component distribution corresponding to musical sound #2

Frequency

Sound particles are stochastically generated
from a weighted sum of component distributions

Frequency

Fig. 2. Mixture modeling of amplitude spectrum.
audio spectra X are stochastically generated from musical notes Z,
several ideas of conventional acoustic models are worth considering.
We explain two major types of acoustic models: mixture models andfactorial models, which can be extended to nonparametric Bayesianmodels by taking the inﬁnite limit of standard Bayesian models when
the space of Z to diverges to inﬁnity.
3.1. Mixture Models
As shown in Fig. 2, the mixture modeling approach assumes that an
amplitude spectrum is generated from a weighted sum of probability distributions that correspond to individual sounds. This meansthat an amplitude spectrum is interpreted as a histogram of “soundparticles” having their own frequencies. If the amplitude value at frequency f is a, we assume that a sound particle of frequency f wasobserved a times. Here, each particle is assumed to be generated
from one of the component distributions. Although this assumption
does not make physical sense, it is known to be useful in practice.
3.1.1. Inﬁnite Latent Harmonic Allocation
One promising idea is to use a Gaussian mixture model (GMM) as
a component distribution. More speciﬁcally, a harmonic structure
consisting of M harmonic partials is explicitly represented by binding component Gaussians to the frequencies of those partials. An
amplitude spectrum consisting of K harmonic sounds is represented
as a mixture of GMMs. An underlying assumption is that each soundparticle is stochastically generated from one of KM Gaussians, i.e.,
one of M partials contained in one of K sounds. This idea is usedby multi-F0 analyzers called PreFEst [4] and HTC [5], in which thevalues of K and M are assumed to be given in advance. The state-ofthe-art analyzer called inﬁnite latent harmonic allocation (iLHA) [2]
is a nonparametric Bayesian version of PreFEst, in which the valuesof K and M are considered to be inﬁnite in theory.
We explain a mathematical formulation of iLHA. Let D be thenumber of frames and regard the observed spectra X as a set of frequencies observed over those frames, i.e., X = {X 1 , · · · , X D }.
X d = {xd1 , · · · , xdNd } is a set of frequencies observed at framed, where Nd is the number of sound particles at that frame. Notethat the value of xdn is represented on a logarithmic scale [cents].
Let Z = {Z 1 , · · · , Z D } and Z d = {z d1 , · · · , z dNd } be the corresponding latent variables, where z dn is a KM -dimensional vector
in which only one entry, zdnkm , takes one and the others take zero
when frequency xdn is generated from partial m (1 ≤ m ≤ M ) of
source k (1 ≤ k ≤ K). Therefore, the space of Z is deﬁned over K
sounds and M partials. Given Z, the likelihood of X is deﬁned aszdnkmp(X|Z, μ, Λ) =N xdn μk + 1200 log2 m, Λ−1
(1)
k
dnkm

where μk is the F0 [cents] of harmonic sound k and Λk indicates adegree of sharpness of harmonic partials on the logarithmic scale. A

The values of amplitude are stochastically generated

Amplitude

Amplitude
Amplitude

(2)

dnkm

The values of amplitude are stochastically generated

Frequency

3.1.2. Inﬁnite Probabilistic Latent Component Analysis
Another promising idea is to use a two-dimensional discrete distribution on the time-frequency plane as a component distribution, which
is factorized as the product of a discrete distribution over frequency
bins and a discrete distribution over frames. These two distributions
respectively correspond to the spectral shape and temporal activationof a sound source. This idea is known as probabilistic latent component analysis (PLCA) [6] and is useful for blind source separation. A
nonparametric Bayesian variant [7] was recently proposed by letting
the number of sound sources, K, diverge to inﬁnity.
Here we explain a mathematical formulation of PLCA. Let D
and F respectively be the numbers of frames and frequency bins.
The observed spectra X can be regarded as a set of frame-bin pairsof sound particles contained in the spectra, i.e., X = {x1 , · · · , xN },where N is the total number of particles. xn is a DF -dimensional
vector in which only one entry, xndf , takes one and the others takezero when the frame and frequency bin of the particle are equal to d
and f . Let Z = {z 1 , · · · , z N } be the corresponding latent variables,
where zn is a K-dimensional vector in which only one entry, znk ,takes one and the others take zero when xn is generated from sourcek (1 ≤ k ≤ K). Therefore, the space of Z is deﬁned over K soundsources. Given Z, the likelihood of X is deﬁned asp(X|Z, ξ, η) =(ξkd ηkf )znk xndf
(3)
nkdf

5355

Frequency

Prob. density

Amplitude

where πdk is a mixing ratio of sound k at frame d and τkm is a mixing ratio of partial m of sound k. Although this model is mathematically convenient for pure acoustic modeling, a musically-meaningfullanguage model over Z should be developed in the future for unsupervised music understanding. To formulate a Bayesian model, we
put conjugate priors over unknown parameters π, τ , μ, and Λ. Morespeciﬁcally, we use K-dimensional and M -dimensional Dirichletpriors as p(π)p(τ ) and use Gauss-Gamma priors as p(μ, Λ). Given
the observed data X, the posterior distribution p(Z, π, τ , μ, Λ|X)
can be computed by using a VB method.
To derive iLHA, we take the limit of p(Z|π, τ )p(π)p(τ ) as K
and M approach inﬁnity. π d contains an inﬁnite number of mixing
ratios {πd1 , · · · , πd∞ } that still sum to unity. p(π d ) is an inﬁnitedimensional Dirichlet distribution, which is known to be equivalent
to a Dirichlet process (DP). A recursive generative process of π d is
known as the stick-breaking process, where the values of π d tend to
decrease exponentially. Since in practice most values are too small, alimited number of harmonic sounds can appear in the observed data
X. A similar discussion can be applied to τ .
Alternatively,
we can consider a marginal likelihood given byp(Z)= (Z|π, τ )p(π)p(τ )dπdτ , in which we do not need to directly deal with inﬁnite-dimensional parameters π and τ . Although
the space of Z is inﬁnite in theory, limited kinds of harmonic sounds
and partials appear in the ﬁnite data X. A recursive generative process of Z is known as the Chinese restaurant process (CRP).
In our experiments using eight piano pieces of jazz and classicalmusic [2], the frame-level accuracy of F0 estimation was 80.9%. The
accuracy of PreFEst, on the other hand, was 78.5% even though the
Dirichlet hyperparameters were carefully tuned. However, a limitation of iLHA is that the number of active sources (effective K) tends
to be overestimated because the GMM is an oversimpliﬁed model ofreal harmonic sounds that contain not only harmonic partials but also
noise components. This could be overcome by fusing iLHA with alanguage model to suppress unlikely note combinations.

Observed
amplitude

simple mixture model over Z is then formulated as follows:(πdk τkm )zdnkmp(Z|π, τ ) =Prob. density

Multiple amplitude spectra are overlappedin a weighted manner and noise is added

Frequency

Fig. 3. Factorial modeling of amplitude spectrum.
where ξkd is the probability that a sound particle of source k is generated from frame d and ηkf is the probability that a sound particleof source k is generated from frequency bin f . A simple mixture
model over Z is then formulated as follows:
 z
ωknk
(4)
p(Z|ω) =nk

where ωk is a mixing ratio of sound k. Bayesian PLCA is easily formulated by using Dirichlet priors as p(ξ)p(η)p(ω). Nonparametric
Bayesian treatment is also straightforward because p(ω) is equivalent to a DP when the value of K approaches inﬁnity.
3.2. Factorial models
As shown in Fig. 3, the factorial modeling approach assumes that an
amplitude spectrum is approximated by a weighted sum of multivariate random variables that correspond to individual sounds. Nonnegative matrix factorization (NMF) [8] has been widely used among
various factorial models such as principal component analysis (PCA)
and independent component analysis (ICA). NMF regards a timefrequency spectrogram as a nonnegative matrix that can be factorized
into the product of two nonnegative matrices, i.e., a set of spectral
bases and a set of temporal activations.
Several nonparametric Bayesian models of NMF have been formulated by letting the number of spectral bases diverge to inﬁnity.
Hoffman et al. [9] proposed the GaP-NMF, which is an inﬁnite extension of the Itakura-Saito divergence NMF [10]. Nakano et al. [11]formulated another GaP-NMF based on the Kullback-Leibler divergence NMF [8] and further extended it to allow each spectral basis
to temporally vary according to an inﬁnite hidden Markov model.
4. PROBABILISTIC LANGUAGE MODELSThe goal of grammar induction is to discover structural patterns S
from musical notes Z for evaluating the structural appropriatenessof those notes in terms of the likelihood p(Z|S). Conventionally,
we are required to deﬁne as S a ﬁxed and ﬁnite vocabulary of structural patterns such as chord types (what note combinations should beconsidered) and chord progressions (how many consecutive chords
should be considered). However, appropriate deﬁnitions heavily depend on the structural complexity of target music data.
To solve this problem, we need to build nonparametric Bayesianmodels by using a prior p(S) over an inﬁnite number of structural
patterns. We explain two types of language models: chain-structured
models and tree-structured models, which can be extended to nonparametric Bayesian models when the space of S diverges to inﬁnity.
4.1. Chain-structured Models
As shown in Fig. 4, the chain-structured modeling approach assumes
that musical notes or chords vary according to Markovian dynamics.
In standard chord progression analysis, n-gram models have been

1-gram
C:maj
2-gramG:7

C:maj
3-gramF:majContext

G:7

C:maj
Next chord

Fig. 4. Chain-structured modeling
Reduction rules
Generate

ReduceFig. 6. Tree-structured modelingn = 1 2 3 4 5 6 7 8 9 10
C:maj
G:maj
A:min
A:min
F:maj7
F:maj6
C:maj
G:maj
F:maj
C:maj
C:maj
G:maj
A:min
A:min
F:maj7
F:maj6
C:maj
G:maj
F:maj
C:maj

which is originated from the Schenkerian theory, where the conﬂicts
between multiple reduction patterns are solved in an ad-hoc manner.
Gilbert and Conklin [16] used a probabilistic context-free grammar(PCFG) for inferring a reduction tree from a melody line by regarding musical notes as words. Kirlin and Jensen [17] proposed a similar tree model for Schenkerian analysis. In the ﬁeld of CL, Liang etal. [18] proposed a nonparametric Bayesian PCFG that can instantiate an unbounded number of reduction patterns as needed. This ishighly promising for inducing a music grammar p(S) itself.
5. CONCLUSION

Fig. 5. Inﬁnity-gram modeling for chord sequencesused for representing sequences of conventional chord labels (e.g.,
C major and D minor) by assuming each chord to depend on a context consisting of n−1 preceding chords [12]. This idea was takenfrom the ﬁeld of computational linguistics (CL), in which n-grammodels are often used for representing sequences of words. For example, Teh [13] proposed a hierarchical Pitman-Yor language model(HPYLM) as the ﬁrst generative model of n-grams. Mochihachi and
Sumita [14] proposed a variable-order Pitman-Yor language model(VPYLM or inﬁnity-gram model) that allows each word (chord) todepend on an unbounded and variate number of preceding words, as
shown in Fig. 5. However, the space of S is ﬁnite because a vocabulary of words (chord labels) are assumed to be speciﬁed.
We introduce a vocabulary-free inﬁnity-gram model p(Z|S) [3]for sequences of simultaneous musical notes Z. This model is an extended version of the VPYLM and does not force us to specify thevalue of n and deﬁne a limited vocabulary of chord labels. The spaceof S is deﬁned over inﬁnitely many kinds of note combinations. A
key building block is the Pitman-Yor process (PY), which is a priordistribution over distributions. Let d and θ be positive scalars andG0 be any distribution. The PY is represented as G ∼ PY(d, θ, G0 ),where d and θ are discount and strength parameters and G is a random distribution. The larger the value of θ is, the more likely it isthat G is similar to G0 . Our n-gram model is obtained by layering
PYs in a hierarchical Bayesian manner. Suppose we have an n-gram
distribution Gu over S, where u is a context of length n − 1. An
n−1-gram distribution Gu∗ given the shortened context u∗ is somewhat similar to Gu . Here Gu is assumed to be drawn from a PY as
Gu ∼ PY(dn , θn , Gu∗ ), where dn and θn are parameters speciﬁc ton. Such a process is deﬁned recursively. Finally, the unigram distribution Gφ is given by Gφ ∼ PY(d0 , θ0 , G0 ), where G0 is a global
base measure (0-gram distribution) over S. An important feature of
our model is that G0 itself is represented as a generative model p(S)that evaluates how likely musical notes are to occur simultaneously.
In our comparative experiments using Beatles songs [3], the perplexity obtained with our model was 14.6, which was signiﬁcantly
better than that obtained with the VPYLM (15.8).
4.2. Tree-structured Models
As shown in Fig. 6, the tree-structured modeling approach assumes
that a temporal sequence of musical notes can be hierarchically clustered according to reduction patterns S. This concept is inﬂuenced
by the Schenkerian theory. Hamanaka et al. [15] proposed a computational model for the generative theory of tonal music (GTTM),

5356

This paper presented a hierarchical Bayesian framework for unsupervised music understanding. Our ultimate goal is to estimate musicalnotes from music audio signals and simultaneously induce structural
patterns from the transcribed notes in an unsupervised way. In initialsteps, we proposed state-of-the-art nonparametric Bayesian acoustic
and language models. Future work includes fusing these models intoa uniﬁed model. We consider exact Bayesian inference to be computationally tractable by alternately optimizing acoustic and language
models with the Gibbs sampling method. In addition, to train a datadriven language model of music, our framework is expected to makeuse of a huge amount of music audio signals on the web even thoughthose signals are not given any ground-truth transcriptions.
6. REFERENCES[1] N. Hjort, C. Holmes, P. Müller, and S. Walker, Eds., Bayesian Nonparametrics, Cambridge University Press, 2010.
[2] K. Yoshii and M. Goto, “A nonparametric Bayesian multipitch analyzer based on inﬁnite latent harmonic allocation,” IEEE Trans. on
ASLP, vol. 20, no. 3, pp. 717–730, 2012.
[3] K. Yoshii and M. Goto, “A vocabulary-free inﬁnity-gram model for
nonparametric Bayesian chord progression analysis,” ISMIR, 2011.
[4] M. Goto, “A real-time music scene description system: PredominantF0 estimation for detecting melody and bass lines in real-world audio
signals,” Speech Communication, vol. 43, no. 4, pp. 311–329, 2004.
[5] H. Kameoka, T. Nishimoto, and S. Sagayama, “A multipitch analyzer
based on harmonic temporal structured clustering,” IEEE Trans. on
ASLP, vol. 15, no. 3, pp. 982–994, 2007.
[6] M. Shashanka, B. Raj, and P. Smaragdis, “Probabilistic latent variable
models as non-negative factorizations,” Computational Intelligence
and Neuoscience, vol. 2008, 2008.
[7] M. Hoffman, D. Blei, and P. Cook, “Finding latent sources in recorded
music with a shift-invariant HDP,” DAFX, 2009.
[8] D. Lee and H. Seung, “Algorithms for non-negative matrix factorization,” NIPS, 2000, pp. 556–562.
[9] M. Hoffman, D. Blei, and P. Cook, “Bayesian nonparametric matrix
factorization for recorded music,” ICML, 2010, pp. 439–446.
[10] C. Févotte, N. Bertin, and J.-L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music
analysis,” Neural Computation, vol. 21, no. 3, pp. 793–830, 2009.
[11] M. Nakano, J. Le Roux, H. Kameoka, T. Nakamura, N. Ono, and
S. Sagayama, “Bayesian nonparametric spectrogram modeling basedon inﬁnite factorial inﬁnite hidden Markov model,” WASPAA, 2011.
[12] R. Scholz, E. Vincent, and F. Bimbot, “Robust modeling of musical chord
sequences using probabilistic N-grams,” ICASSP, 2009, pp. 53–56.
[13] Y. W. Teh, “A Bayesian interpretation of interpolated Kneser-Ney,”
Tech. Rep. TRA2/06, National University of Singapore, 2006.
[14] D. Mochihashi and E. Sumita, “The inﬁnite Markov model,” NIPS,
2007, pp. 1017–1024.
[15] M. Hamanaka, K. Hirata, and S. Tojo, “FATTA: Full automatic timespan tree analyzer,” ICMC, 2007, pp. 153–156.
[16] E. Gilbert and D. Conklin, “A probabilistic context-free grammar for
melodic reduction,” IJCAI, 2007, pp. 83–94.
[17] P. Kirlin and D. Jensen, “Probabilistic modeling of hierarchical music
analysis,” ISMIR, 2011, pp. 393–398.
[18] P. Liang S. Petrov, M. I. Jordan, and D. Klein, “The inﬁnite PCFG
using hierarchical Dirichlet processes,” EMNLP, 2007, pp. 688–697.