Infinite Latent Harmonic Allocation:

A Nonparametric Bayesian Approach to Multipitch Analysis
Kazuyoshi Yoshii and Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST)We need a methodology to deal withuncertainty inherent in music analysis

Example: F0 detection from polyphonic sounds
It is often difﬁcult
to make binary
decision on
the existence ofeach musical note

Modestly conﬁdent
Hardly conﬁdent
Absolutely conﬁdent

III. Conventional Parametric Models
Finite GMMs for monophonic spectra

PreFEst [Goto1999] HC, HTC [Kameoka2004, 2007]
Prob.
density

Each Gaussian corresponds
to one of harmonic partials

 Frequency

F0[cent]

Posterior distributions of all unknown variables(not limited to parameters) should be estimated
The values of F0s?

How many F0s?

How optimize prior distributions?

Parameters

Complexity

Hyperparameters

Robustness

Maximum likelihood
estimation (ML)Point
estimates

Manually
speciﬁed

Nothing

Bad

Maximum a posteriori
estimation (MAP)

Pointestimates

Manually
speciﬁed

Manually
speciﬁed

ModestClassical Bayesianestimation

Posterior
distributions

Manually
speciﬁed

Manually
speciﬁed

Good(1) Nonparametric
Bayesian estimationPosterior
distributions

Posterior
distributions――――

Excellent(2) Hierarchical
Bayesian estimation

Posteriordistributions――――

Posterior
distributions

Excellent

V. Inﬁnite Latent Harmonic Allocation (iLHA)

Key feature (1): Nonparametric Bayesian formulation

The Dirichlet process (DP) prior can generate the inﬁnite
number of mixing weights and lead them to become “sparse”…

Most weights are almost equal to 0

Each GMM corresponds
to a harmonic structure
Mix

Frequency[cent]
Assumption:
Amplitudes are additive

GMMs

Frequency[cent]

Stick-breaking
construction:
Recursively break the stick of length 1
The average splitting ratio isSmall

Large

Short-tailed

Long-tailed

…

Prob.
Prob.
density density

Nested ﬁnite GMMs for polyphonic spectra

II. “Completely” Bayesian Treatment

Mixingweight

I. Why Take Bayesian Approach?

Prior:How optimize inﬂuential hyperparameter?

Key feature (2): Hierarchical Bayesian formulationin frame

Mixing weight of harmonic partial

in sound source

IV. Proposed Nonparametric Models

Nested inﬁnite GMMs for polyphonic spectra

Model complexities are considered to be inﬁnite

Hyperprior:: Shape and rate parameters

Generate the inﬁnite numberof mixing weights (probabilities)of harmonic partials
Select one ofpartials in one ofGenerate frequency
Manually tuned“Nonparametric”
means thatthe size ofparameter
space isneither ﬁxed
nor limited(it does notmean that
there are noparameters)

Completely automatedConventional

Proposed#(sound sources))

Fixed (K)Inﬁnite#(harmonic partials)

Fixed (M)InﬁnitePrior on mixing weights
of sound sources

Nothing

Noninformative
hyperprior+ Hierarchical Dirichlet
process (HDP)

Prior on mixing weights Dirichlet
distributionof harmonic partials
Training method

MAP estimation

Noninformative
hyperprior+ Dirichlet process (DP)Bayesian estimation

VII. Conclusion

Our contributions
We proposed an ultimate mixture-model-based
method for multipitch analysis
This is the ﬁrst attempt to apply the nonparametric
Bayesian framework to multipitch analysis
Future directions
We plan to use this framework in a wide range ofapplications such as content-based clustering ofmusical pieces and musical structure analysis

Uniform
distributionProb.
densityMixing weight of sound source

The concentration parameter (hyperparameter) of the DP
is assumed to follow a noninformative Gamma hyperprior

General forms

sourcesfrom the selected partial

Special caseThis naturally represents
our situation that we havelittle knowledge on

How train the model?
Gamma

DPMultinomial

GaussianN

Collapsed Variational Bayes

D Full joint distribution

KGamma

K →∞
M →∞

DP

Marginal distribution
DP

Gaussian-Wishart

True posterior

Variational posterior

Gamma

Generate the inﬁnite numberof mixing weights (probabilities)
of sound sources

VI. Comparative Evaluation

Data: Polyphonic audio of piano/guitar performances
6 pieces from RWC-MDB-J-2001: Jazz Music
2 pieces from RWC-MDB-C-2001: Classical Music23 [s] excerpted from the beginning of each piece
Frequency analysis: Gabor wavelet transform
Evaluation criterion: Frame-level F-measures
The completely automated method (iLHA) yieldedvery competitive results against carefully tuned
conventional methods (PreFEst and HTC)

Iterative approximation via the VB-EM algorithm
Hand-tuned prior+ Temporal modeling+ MAP estimation
Hand-tuned prior+ MAP estimation

Noninfo. hyperprior+ Nonparametric Bayes+ Bayesian estimation
Noninfo. prior+ Bayesian est.

PreFEstHTC

LHAiLHAJ No.1

75.8

79.0

70.7

82.2

J No.2

78.5

78.0

69.1

77.9

J No.6

70.4

78.3

49.8

71.2

J No.7

83.0

86.0

70.2

85.5

J No.8

85.7

84.4

55.9

84.6

J No.9

85.9

89.5

68.9

84.7

C No.30

76.0

83.6

81.4

81.6

C No.35

72.8

76.0

58.9

79.6

Total

79.4

82.0

65.8

81.7