A Generative Product-of-Filters Model of Audio
Authors: Dawen Liang, Mathew D. Hoffman, Gautham Mysore
ICLR 2014

Abstract: We propose the product-of-filters (PoF) model, a generative model that decomposes audio spectra as sparse linear combinations of 'filters' in the log-spectral domain. PoF makes similar assumptions to those used in the classic homomorphic filtering approach to signal processing, but replaces hand-designed decompositions built of basic signal processing operations with a learned decomposition based on statistical inference. This paper formulates the PoF model and derives a mean-field method for posterior inference and a variational EM algorithm to estimate the model's free parameters. We demonstrate PoF's potential for audio processing on a bandwidth expansion task, and show that PoF can serve as an effective unsupervised feature extractor for a speaker identification task.


Peer review:
[Review]:
Summary: This paper describes a simple generative model for audio spectra from a single sound source. It explains them as products of spectral bases, i.e., sums of log-spectral bases.  The paper describes a mean field variational approximation for the posterior of the activations of each basis given an observed spectrum, and a variational EM algorithm for learning the model parameters.  It applies the model to performing bandwidth extension for unknown talkers and frame-based speaker identification, outperforming reasonable baseline approaches in both.

Novelty and quality: As far as I know, this method is novel.  The background literature review correctly points to the connections with traditional methods of homomorphic filtering and non-negative matrix factorization.  While it is not entirely clear how widely this method can be applied, especially with the high current computational cost of learning the model parameters compared to a traditional fixed basis like mel frequency cepstral coefficients (MFCCs), it does significantly outperform baseline systems on the two experimental tasks described.  Specifically, real sounds only rarely involve a single source, but, as mentioned in the paper, this model could provide a prior on each source in a multi-source mixture.

Pros: 
* Interesting model 
* Shows good experimental results 
* Well written

Cons: 
* Some mathematical details could be spelled out more explicitly 
* Possibly limited applicability


Minor comments:

Page 2: 'Another attractive feature is the symmetry between the excitation signal e[n] and the impulse response h[n] of the vocal-tract filter—convolution commutes.'

Please explain why this is an attractive feature of the model. It seems to be an additional nuisance degree of freedom in the model.

Page 4: The definition of q(a_{tl}) uses a_{tl}, where everything else uses a_{lt}. Is this intentional?

Page 6: 'The periodic “excitation” filters tend to be used more rarely, ... the excitation signal generated by a speaker’s vocal folds.'

This statement seems to assume that multiple excitation bases cannot be combined into a valid excitation, while multiple vocal tract shape bases can be combined to create a valid vocal tract shape.  Can you prove or demonstrate that this is the case?

Page 6: 'We can approximate this posterior using the variational inference algorithm from section 4.1'

Can you show this more explicitly?