Stopping Criteria in Contrastive Divergence: Alternatives to the Reconstruction Error
Authors: David Buchaca, Enrique Romero, Ferran Mazzanti, Jordi Delgado
ICLR 2014

Abstract: Restricted Boltzmann Machines (RBMs) are general unsupervised learning devices to ascertain generative models of data distributions. RBMs are often trained using the Contrastive Divergence learning algorithm (CD), an approximation to the gradient of the data log-likelihood. A simple reconstruction error is often used to decide whether the approximation provided by the CD algorithm is good enough, though several authors (Schulz et al., 2010; Fischer & Igel, 2010) have raised doubts concerning the feasibility of this procedure. However, not many alternatives to the reconstruction error have been used in the literature. In this manuscript we investigate simple alternatives to the reconstruction error in order to detect as soon as possible the decrease in the log-likelihood during learning.


Peer review:
[Review]:
Hi David,

After briefly going through your paper, I have a number of comments on it. 

- The proposed stopping criterion itself is not so novel, in my opinion. Essentially, it is a general framework in which, for instance, CD minimizes, where $y^{(i)}$ is obtained by a single-step Gibbs sampling from $x^{(i)}$. In fact, if $y^{(i)}$ is one step of persistent MCMC chain, the stopping criterion corresponds to the approximate maximum likelihood criterion used by PCD.  In short, the proposed criterion is, in fact, what a proper MLE should minimize.

- The novelty is, however, in the proposed ways to choose $y^{(i)}$. The authors proposed two alternatives. Unfortunately (as somewhat expected) the first method of selecting random points does not work, which is only natural as if it did, why would anyone use MCMC to estimate the model statistics of RBM? :) The second alternative of using the visible samples conditioned on the *flipped* hidden states seems to work better, but the authors do not provide any good justification for this choice. 

- The experiments show that the second choice is superior, but since the experiments are rather small-scale, it is difficult to make a solid conclusion from them. Also, the proposed method seems to only work with CD_1 (see Fig. 4), but the authors' explanation on possible reasons for this is not clear. 

- One thing which is clearly missing from the experiments is the comparison of the proposed criterion against the actual function CD minimizes ($E_D left[ log p (x^{(i)}) - log p(y^{(i)}) ight]$, where $y^{(i)}$ is the one-step Gibbs sample starting from $x^{(i)}$.) as well as the approximate likelihood (the same thing, but use samples from persistent chains as $y^{(i)}$'s). Computing these things should be no more expensive than computing the proposed criterion, and may give better indication of how log-likelihood evolves over time. This will be  interesting to see. 

- Of course, one last thing is, why anyone would try to train an RBM to be a good generative model (maximizing the log-likelihood) using CD in the first place, while PCD (with possibly other MCMC algorithms than Gibbs) is available.

Despite these seemingly negative comments, I enjoyed reading the paper, since it is clearly written and well motivated. A few more experiments showing that the proposed methods of choosing $y^{(i)}$ are better than other possible choices (e.g., using Gibbs sampling) would make the paper much stronger.

- Cho 

P. S. I wouldn't mind, if you cited my paper (IJCNN 2013) next to (Desjardins et al., 2010) where I also proposed PT. :)