Title: Boosted Generative Models
Authors: Aditya Grover
Stefano Ermon
Venue: ICLR 2017
Keywords: Theory
Deep learning
Unsupervised Learning

Abstract: We propose a new approach for using boosting to create an ensemble of generative models, where models are trained in sequence to correct earlier mistakes. Our algorithm can leverage  many existing base learners, including recent latent variable models. Further, our approach allows the ensemble to leverage discriminative models trained to distinguish real data from model generated data. We show theoretical conditions under which incorporating a new model to the ensemble will improve the fit and empirically demonstrate the effectiveness of boosting on density estimation and sample generation on real and synthetic datasets.


Review discussion:
Could the authors provide a more detailed comparison to the work of Tu (2007) ? There seems to be a significant overlap with Sec. 2.2 (including Theorem 2 and its proof), which is not fairly represented in the related works section IMO. I believe more credit and transparency in this regard would benefit the paper, without adversely impacting novelty.
As for the experimental section, what is the added benefit of mixing various base-learners ?
Even though the method allows for it, there is no clear rationale for doing so. Unfortunately, this seems to come at the expense of careful evaluation of each approach. As it stands, the experimental section does not convincingly answer the following questions: which approach of DiscGBM or GenGBM is better ? In which condition does boosting outperform the ML baselines ? Table 1 seems to imply there may a win in terms of training time, but this seems unconvincing without likelihoods estimates, convergence curves, etc.
In particular, the samples obtained by the baseline models on MNIST seem poor compared to what can be found in the literature.
Theorem 2 is a direct consequence of the unsupervised-as-supervised learning approach proposed by Friedman et al. (2001) [Sec. 14.2.4 in the book] which has been used by many subsequent works including Tu (2007). The theoretical analysis and algorithm proposed in Tu (2007) differ from the one we propose in the following ways:
1. Tu (2007) does not account for the more practical case where we have imperfections in learning intermediate models. We account for these imperfections by assigning weights (i.e., alpha's) to intermediate models.
Furthermore, Theorem 1 in our work is for the more general case where alpha's lie between 0 and 1 (both inclusive) and the analysis of Tu (2007) falls out as a special case (all alpha's = 1).
2. Theorem 1 in Tu (2007) does not clearly state the hypothesis/conditions under which the claimed statement holds. If we assume the assumptions to be “Z_k<=1 and  \int p+(x) log odds >=1”, as mentioned after the proof, these together are only a sufficient condition. In our notation, this sufficient condition translates to E_{q_{t-1}}[h] <=1 and E_p[log h] >=0. In the setting considered in Tu (2007) where all \alpha’s=1, our sufficient condition (Theorem 1 in our work, condition (1)) is also necessary, and strictly stronger as the following simple example demonstrates. Let p and q_{t-1} be degenerate distributions assigning all probability to a single point, let’s say x_p and x_q i.e., p(x_p)=1 and 0 elsewhere, q_{t-1}(x_q)=1 and 0 elsewhere. Set h(x_p) = (0.5+\epsilon_p)/(0.5-\epsilon_p) and h(x_q)= (0.5+\epsilon_q)/(0.5-\epsilon_q) for any 0< \epsilon_q <= \epsilon_p < 0.5.  The sufficient condition of Tu (2007) is unsatisfied, because E_{q_{t-1}}[h] >1.
However, our sufficient and necessary condition (in our Theorem 1) is satisfied.
3. Algorithmically, the intermediate models in our case can be generators/discriminators whereas Tu (2007) considers only discriminators. Even for the discriminator version, we have a discussion on the effect of different type of classifiers such as random and adversarial classifiers (Corollary 1) which is absent in Tu (2007).
Experiments
1. Different base learners have different inductive biases, and these biases capture different patterns in the observed data.
For instance, discriminators such as CNNs can account for translational invariance in images, latent variable models such as VAEs/RBMs are good for learning compact continuous/discrete latent representations.
Furthermore, mixing models helps in task generalization as evident in our experiments;
while the VAE->VAE and RBM->RBM combinations are among the best performing models for sample generation and unsupervised feature learning respectively, they turn out to be among the weakest models for unsupervised feature learning and sample generation respectively. The hybrid models allow us to interpolate between extremes and derive generative models that perform well across different tasks.
2. In line with the above comment, it is hard to make definitive statements about either of GenBGM or DiscBGM being better than the other without considering the specific task and dataset.
However, we reiterate some observations that we expect to generally hold in practice.
First, DiscBGM consists of a computationally expensive of sampling from the previous BGM distribution unlike GenBGM which requires simply a reweighting of the data distribution.
Second, the synthetic experiments for density estimation show that while both can algorithms can correct for model misspecification, GenBGM leans slightly towards underfitting (Fig. 1(h)) whereas DiscBGM can be hard to generalize in the absence of appropriate regularization (Fig. 1(k)).
3. Note that the samples obtained by baseline models could be poor due to model misspecification and/or computational budget for optimization. The state-of-the-art MNIST samples from literature are generated using highly expressive models trained for a long duration. We use simple baseline models to leave room for improvement for clear qualitative evaluation. Our experiments show that the boosting approach can be used to build highly expressive models without incurring any significant computation overheads. The baseline models were indeed run up to convergence with regularization imposed by early stopping based on monitoring of cross-validation loss (which is routinely the procedure followed in literature) whereas the individual models in BGM sequences were stopped much earlier before convergence (to balance overall computation time) putting them at a disadvantage.
Furthermore, we didn’t observe the sample quality from boosted sequences to match the BGM sample qualities even with a more expressive baseline model with twice the hidden layers trained up to convergence.
Hence, naively increasing the computational budget and/or model capacity by training highly expressive models is empirically observed to be not as effective as boosting.
The paper proposes two approaches to boosting generative models, both based on likelihood ratio estimates. The approaches are evaluated on synthetic data, as well as on MNIST dataset for the tasks of generating samples and semi-supervised learning.
While the idea of boosting generative models and the proposed methods are interesting, the reviewer finds the experiments unconvincing for the following reasons.
1. The bagging baseline in section 3.1 seems to be just refitting a model to the same dataset, raising the probability to power alpha, and renormalizing. This makes it more peaked, but it's not clear why this is a good baseline. Please let me know if I misunderstood the procedure.
2. The sample generation experiment in section 3.2 uses a very slowly converging Markov chain, as can be seen in the similarity of plots c and f, d and g, e and h. It seems unlikely therefore that the resulting samples are from the stationary distribution. A qualitative evaluation using AIS seems to be necessary here.
3. In the same section the choices for alphas seem quite arbitrary - what happens when a more obvious choice of alpha_i=1 for all i is made?
4. It seems hard to infer anything from the semisupervised classification results reported: the baseline RBM seems to perform as well as the boosted models.
The work is mostly clearly written and (as far as the reviewer knows) original.
Thanks for your helpful comments. Please refer to the common rebuttal posted for response to questions regarding MNIST experiments (points 2 and 3).
1. Yes, the bagging procedure described is correct. The purpose of including bagging as a baseline was to demonstrate that naively increasing model capacity by adding new models to the ensemble is not very effective in correcting for model misspecification, unlike boosting which involves a reweighting step in the GenBGM version or classifier training for the DiscBGM version. A similar approach (called bagging) competes with boosting for the supervised case. For brevity, we have removed this baseline from the latest version.
4. The experiments on semi-supervised classification have been removed in the latest version since the gain in accuracy with the current algorithm is unconvincing for a few cases. For completeness, we would like to clarify the purpose of including them earlier was to demonstrate a) the computational gains that boosted generative models can provide on this task (for example, the training time of RBM->RBM was slightly more than half of the baseline RBM model yet it was able to match the performance of the baseline RBM model) b) improving models that are weaker at this task due to different inductive biases (for example, VAE->RBM gives better accuracy than the baseline VAE). Based on our preliminary experiments, we believe this task is an important use case of boosted generative models.
However, in the absence of sufficient empirical evidence, we defer improved algorithms for semi-supervised classification based on the boosting framework to future work.
The authors propose two approaches to combine multiple weak generative models into a stronger one using principles from boosting. The approach is simple and elegant and basically creates an unnormalized product of experts model, where the individual experts are trained greedily to optimize the overall joint model. Unfortunately, this approach results in a joint model that has some undesirable properties: a unknown normalisation constant for the joint model and therefore an intractable log-likelihood on the test set; and it makes drawing exact samples from the joint model intractable. These problems can unfortunately not be fixed by using different base learners, but are a direct result of the product of experts formulation of boosting.
The experiments on 2 dimensional toy data illustrate that the proposed procedure works in principle and that the boosting formulation produces better results than individual weak learners and better results than e.g. bagging.
But the experiments on MNIST are less convincing: Without an undisputable measure like e.g. log-likelihood it is hard to draw conclusions from the samples in Figure 2; and visually they look weak compared to even simple models like e.g. NADE.
I think the paper could be improved significantly by adding a quantitative analysis: investigating the effect of combining undirected (e.g. RBM), undirected (e.g. VAE) and autoregressive (e.g. NADE) models and by measuring the improvement over the number of base learners.
But this would require a method to estimate the partition function Z or estimating some proxy.
Thanks for your helpful comments. Please refer to the common rebuttal posted for response to questions regarding MNIST experiments.
Regarding the concerns with the unnormalized final density, we highlight that probabilistic models frequently make a trade-off between expressiveness and tractability. Simple models such as NADE and mixture of Bernoullis make assumptions that lend tractability but are not very flexible in modeling complex structure in the data.
On the other hand, latent variable models such as VAEs and RBMs have great expressive power but need to approximate intractable integrals. Boosted generative models also have a computationally intractable partition function  with the trade-off made in expressiveness just like VAEs, RBMs, GANs (where even the unnormalized log-likelihood cannot be directly computed), etc. We argue that intractability is however not a deal-breaker since a). many use cases of generative models (such as sampling, unsupervised feature learning) do not require the partition function, b). wherever required, there are some generic techniques available for estimating the partition function.
This paper extends boosting to the task of learning generative models of data. The strong learner is obtained as a geometric average of “weak learners”, which can themselves be normalized (e.g. VAE) or un-normalized (e.g. RBMs) generative models (genBGM), or a classifier trained to discriminate between the strong learner at iteration T-1 and the true data distribution (discBGM). This latter method is closely related to Noise Contrastive Estimation, GANs, etc.
The approach benefits from strong theoretical guarantees, with strict conditions under which each boosting iteration is guaranteed to improve the log-likelihood. The downside of the method appears to be the lack of normalization constant for the resulting strong learner and the use of heuristics to weight each weak learner (which seems to matter in practice, from Sec. 3.2). The discriminative approach further suffers from an expensive training procedure: each round of boosting first requires generating a “training set” worth of samples from the previous strong learner, where samples are obtained via MCMC.
The experimental section is clearly the weak point of the paper. The method is evaluated on a synthetic dataset, and a single real-world dataset, MNIST: both for generation and as a feature extraction mechanism for classification. Of these, the synthetic experiments were the clearest in showcasing the method. On MNIST, the baseline models are much too weak for the results to be convincing. A modestly sized VAE can obtain 90 nats within hours on a single GPU, clearly an achievable goal.
Furthermore, despite arguments to the contrary, I firmly believe that mixing base learners is an academic exercise, if only because of the burden of implementing K different models & training algorithms. This section fails to answer a more fundamental question: is it better to train a large VAE by maximizing the elbow, or e.g. train 10 iterations of boosting, using VAEs 1/10th the size of the baseline model ? Experimental details are also lacking, especially with respect to the sampling procedure used to draw samples from the BGM. The paper would also benefit from likelihood estimates obtained via AIS.
With regards to novelty and prior work, there is also a missing reference to “Self Supervised Boosting” by Welling et al [R1]. After a cursory read through, there seems to be strong similarities to the GenBGM approach which ought to be discussed.
Overall, I am on the fence. The idea of boosting generative models is intriguing, seems well motivated and has potential for impact.
For this reason, and given the theoretical contributions, I am willing to overlook some of the issues highlighted above, and hope the authors can address some of them in time for the rebuttal.
[R1] https://papers.nips.cc/paper/2275-self-supervised-boosting.pdf
PROS:
Novel and intriguing idea
Strong theoretical guarantees
CONS:
Resulting boosted model is un-normalized
Discriminator based boosting is expensive, due to sampling via MCMC
Weak experimental section
Thanks for your helpful comments. Please refer to the common rebuttal posted for response to questions regarding MNIST experiments. Additional comments:
1. We have followed up on the reviewer’s suggestion of removing experiments that mix different base learners. We hope this makes the presentation clean.
2. We have restructured the related works section entirely for improved clarity, and also included a discussion on the differences of our framework with “self-supervised boosting”  -- a reference we had missed out earlier and are duly thankful to the reviewer for highlighting. We have also included references to some very recent relevant preprints that came after our work for a more complete and up-to-date discussion.
3. Please read the response to Reviewer 4 below for an argument on why we feel that unnormalized models such as BGMs while certainly being limited are not a deal-breaker.
The bulk of the reviewer (R) comments pertain to the experiments on MNIST that we address collectively here to facilitate a richer, holistic discussion. Any remaining questions raised by the reviewers are addressed individually as follow-ups to the reviews.
We have rewritten the entire experimental section based on fresh experimentation, focusing on aspects that we believe are crucial to resolving the concerns raised by the reviewers. For brevity, we have also removed some portions from the earlier draft that we do not believe are central to the discussion. In the spirit of the constructive feedback loop that ICLR facilitates, we hope the reviewers will be able to read the revised experiments section. Our response to the reviewer comments:
1. A bigger VAE vs. BGMs containing smaller models (R1)
Our earlier submission demonstrated improved performance of BGMs for sampling with gains in computational requirements, but did not directly address the reviewer’s question about model capacity. Our latest submission considers larger baseline models with equal or more capacity than BGM sequences of simpler models (Sec. 3.2.2, Table 2), and shows the superior performance of BGMs for this task (Figure 3) . As before, we also illustrate that the performance gains of BGM sequences are not at the cost of increased training time (Sec 3.2.3, Table 2).
2. Samples from MNIST look weak (R4)
Our experiments on MNIST do not (or even intended to) make any state-of-the-art claims, but instead demonstrate the improvements in computational and statistical performance that simpler models can attain for the task of generating samples as a result of the boosting procedure.
Having said that, we respectfully disagree that the samples produced by BGM sequences look weak keeping in mind that the dataset under consideration is the harder ‘binarized’ version of MNIST.
In fact, although we understand that it is somewhat subjective, we believe the BGM samples look better than NADE samples on the same dataset (Figure 2 (Left) in [2]). Note that we display actual samples produced by our models (without any cherry-picking), and hence, these samples should not be compared with the probabilities from sampling pixels (Figure 2 (Middle) in [2]).
3. Strategies for setting weights (\alpha’s) for intermediate models (R3)
In our earlier submission, we discussed model weights in the theoretical section and listed heuristic strategies for setting them in practice as a future work. Our latest submission includes experimental results and discussion of some heuristic strategies that we propose for both density estimation and sampling. See Sec. 3.1.3 and Appendix A.2.1.
4. Procedure for drawing samples from BGM sequences (R1, R3)
See Appendix A.2.3 for details.
In contrast to results in the previous submission where the independent chains were run from the same start seeds across BGM sequences and samples arranged for easy comparison, the latest results show samples produced from chains that are run independently, initialized with different random start seeds, and have an increased burn-in time of 100,000 samples (10x the previous) to mitigate the possibility of any mixing-related issues. We also observed visually that the samples from any given chain vary over time transforming smoothly across multiple digits within the burn-in time, a further indication of successful mixing.
5. Quantitative estimation of log-likelihoods for sampling (R1, R3, R4)
For evaluating samples, log-likelihoods can be problematic. See [1], Sec. 3.2 for a detailed discussion on why log-likelihoods are not a good proxy for sample quality.
Additionally, generative models such as VAEs, RBMs have intractable likelihoods and hence, resort to their own approximations for generating density estimates. These approximations are known to exhibit different behaviors;
for instance, RBMs estimated using AIS are typically believed to overestimate the log-likelihoods, whereas VAEs make variational approximations that only provide lower bounds -- it is difficult to make definitive statements based on such approximations. We tried AIS on the BGM sequences considered in the paper with about 15,000 intermediate annealing distributions (step-size of 1e-3 between for the first 500 distributions, 1e-4 for the next 4000, and 1e-5 for the remaining where every transition from one distribution to another was implemented using 10 steps of Metropolis-Hastings), but got unrealistically optimistic log-likelihood evaluations that we do not feel confident reporting for the purpose of a meaningful comparison with the lower bounds provided by VAEs.
References:
[1] Lucas Theis, Aaron van den Oord, and Matthias Bethge. “A note on the evaluation of generative models.” In ICLR, 2016.
[2] Hugo Larochelle and Iain Murray. "The Neural Autoregressive Distribution Estimator."  In AISTATS, 2011.
Thanks to the authors for updating the paper. The new experimental section does seem more streamlined, and includes previously missing experimental details. That being said, I stand by my previous score.
Generative models have improved tremendously over the past few years. VAEs allow for a tractable lower-bound, are trivial to sample from and can yield impressive performance on state-of-the-art natural image datasets.
While GANs lose tractability, they similarly allow for ancestral sampling, with impressive performance. Unfortunately, GBMs are both intractable and require MCMC for sampling. These are significant drawbacks. Unfortunately, the paper does not convincingly show that the pros outweigh the cons.

Meta-review:
role: PC
data: comment: The idea of boosting has recently seen a revival, and the ideas presented here are stimulating. After discussion, the reviewers agreed that the latest updates and clarifications have improved the paper, but overall they still felt that the paper is not quite ready, especially in making the case for when taking this approach is desirable, which was the common thread of concern for everyone. For this reason, this paper is not yet ready for acceptance at this year's conference.

Decision: Reject