Greedy Bayesian Posterior Approximation with Deep Ensembles

Abstract: Ensembles of independently trained neural networks are a state-of-the-art approach to estimate predictive uncertainty in Deep Learning, and can be interpreted as an approximation of the posterior distribution via a mixture of delta functions. The training of ensembles relies on non-convexity of the loss landscape and random initialization of their individual members, making the resulting posterior approximation uncontrolled. This paper proposes a novel and principled method to tackle this limitation, minimizing an $f$-divergence between the true posterior and a kernel density estimator in a function space. We analyze this objective from a combinatorial point of view, and show that it is submodular with respect to mixture components for any $f$. Subsequently, we consider the problem of greedy ensemble construction, and from the marginal gain of the total objective, we derive a novel diversity term for ensemble methods. The performance of our approach is demonstrated on computer vision out-of-distribution detection benchmarks in a range of architectures trained on multiple datasets. The source code of our method is made publicly available.

Reviews and author responses:

 I thank the authors for the further clarification. After reading the rebuttal, I still feel positive about this paper, but, honestly, my confidence is low. So, I'd like to further discuss with the other reviewers to reach a consensus.

 Apologies for too many revisions. We found another disturbing typo, which is now fixed.

 Dear all,

We have also fixed a few typos compared to our first revision. 

 Dear Reviewer, 

thanks for the positive feedback! Please find the answers to your questions below.

**Universality of KL-divergence:** Indeed, in the paper we did not make any further investigation of other generators than $f(x) = -\log x$. In this submission, we have already shown the utility of our framework using KL-divergences while providing a general proof that all f-divergences are amenable to submodular optimization. We agree that exploration of other divergences is interesting future work.

**Relation to f-POVI:** Indeed, our paper is related to f-POVI objectives, and [1] makes this connection even more obvious. However, the key difference between our method and f-POVI, is that a) our method works with state-of-the-art training techniques b) has theoretical approximation guarantees c) has $O(1)$ memory complexity. We further highlighted the difference between the methods in the Discussion section.

**More experiments:** We agree that it is always desirable to make more comparisons. However, we believe that we have a very extensive benchmark with four in-domain and 10 out-of-domain datasets. Furthermore, as mentioned in review to dcPg, the cost of the experiments in the paper is already equivalent to roughly \$200,000. 

Finally, we would like to comment about the marginal improvement of the results. All our experiments were run 5 times, and we always reported standard errors over runs when they are greater or equal to $0.01$. The numbers in tables were highlighted in bold only those cases when the mean $\pm$ standard error did not overlap. Thus, the improvement over Deep Ensembles should be considered substantial and significant. 

**References**
1. D'Angelo, Francesco, and Vincent Fortuin. "Repulsive Deep Ensembles are Bayesian." NeurIPS 2021

 Dear Reviewer,

Thanks for providing the comments! Please find our response below.

**Importance of submodularity:** We have added a smoother transition to submodularity in the introduction.

**Mean and mode-seeking behavior:** This is, unfortunately, difficult to quantify for any problem but toy ones, as we generally cannot enumerate modes of the posterior for real settings and as a result do not have ground truth. If the reviewer wishes, we can conduct experiments on randomly generated multimodal distributions and post the results to OpenReview. 

**Quality of approximations:** We agree with the reviewer that our approximations have been used without any additional discussion. However, approximations are required in any Bayesian framework to make the inference tractable outside toy problems. A very recent work on Repulsive Deep Ensembles [1] fortunately makes a connection between training an ensemble and a Bayesian posterior via the Wasserstein Gradient Descent (WGD). In our work, we make very similar approximations to [1], and believe that the future works will find a direct connection between our method and WGD. However, we note that [1] is "very recent work" according to the ICLR reviewer guidelines (see https://iclr.cc/Conferences/2022/ReviewerGuide})

**Scaling to large datasets:** Our method has the same complexity as Deep Ensembles -- $O(M)$. The only difference is that our method is less straightforward to parallelize. In the paper (see discussion), we have already pointed out that there exist parallel algorithms to maximize submodular functions, and that we have chosen to present greedy optimization in this work for simplicity.  It is absolutely feasible to run our method on ImageNet, but we have chosen datasets in this paper that allow for thorough repeated empirical comparisons for statistical significance.  This already represents a very large amount of computation - see also our response to reviewer dcPg regarding comparison to other baselines.

In addition to the point above, we would like to mention that trying to convert methods like e.g. [1] will have $O(M)$ in memory complexity, which makes them absolutely impractical for ImageNet-scale datasets. 


**Adversarial attacks:** Our paper makes no specific claims about adversarial attacks, and we expect that our approach should be no worse than other deep ensemble approaches. Ensembles should be in general more robust to adversarial attacks than point estimates, but this is rather an orthogonal question to those addressed in this submission. If the reviewer is interested in the robustness of the method, we point to Appendix C.4, where we have conducted an evaluation on a distribution shift benchmark.

**References**
1. D'Angelo, Francesco, and Vincent Fortuin. "Repulsive Deep Ensembles are Bayesian." NeurIPS 2021

 **The provided explanations are over-complicated:** We argue that there is a place for purely theoretical, purely empirical, and also "in between" papers at ICLR (see https://iclr.cc/Conferences/2022/ReviewerGuide), and our work should be treated as belonging to the latter category. Furthermore, we argue that mathematical rigor is required in Bayesian Deep Learning, and, furthermore, needed to make progress. We recognize that for some readers the paper might look complex, however, as we have mentioned, our work makes theoretical contributions to the field while still showing that the resulting Bayesian method can work in practice. 

**Figure 1:** We have attached additional code to our repository. Please, take a look at the script generating Figure 1 at https://anonymous.4open.science/r/greedy_ensembles_training-0CD6/experiments/figure1_vis.ipynb. Running this notebook (all cells at once) will generate the results and save them to disk (basic python dependencies needed). 

**Comparisons to other baselines:**
We would like to clarify again that

1. f-POVI type of methods are from a different class, thus there is no need to compare to them (see our points about data augmentation above).
2. Techniques like BatchEnsembles [2] aim to make ensembles more efficient, thus also belong to a different class of methods.
3. Techniques that add more diversity, such as e.g. hyperparameter ensembles [3] do not make substantial improvements over Deep Ensembles on OOD benchmarks. Here, we refer to the results presented in the appendix of [3].
4. The experiments shown in the paper have a cost of almost $200,000 in AWS credits. 

Given the points above, the comparisons presented in our work are not only valid scientifically, but also provide a lot of value for the community due to extensive compute used to generate the models.

**References**
1.  Resnick, Sidney. A probability path. Springer, 2019.
2. Wen, Tran, Ba. BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning, ICLR 2020
3. Wenzel et al. Hyperparameter Ensembles for Robustness and Uncertainty Quantification, NeurIPS 2021.

 Dear Reviewer,

We appreciate the provided comments, although we are very surprised with such a low grading of our submission. We hope that after addressing the raised concerns in the rebuttal, you will reconsider the score. 

**Contextualization:** As ICLR allows for one extra page after rebuttal, we have added a sentence to the introduction that motivates the use of submodularity. Furthermore, we urge the reviewer to look at Section 2.1, which rigorously defines the problem state from a mathematical standpoint, adding more context.

**Mathematical notation:** The reviewer raised concerns about the mathematical notation in e.g. Definition 1. We urge the reviewer reconsidering this point, as the foundations of modern probability, introduced already in the 1930s by Kolmogorov, are based on measure theory, and thus the use of set theory notation should not be questioned in Bayesian inference, see e.g. [1]. 

However, after reading this comment, we have noticed some abuse of notation, which might have eventually caused concerns and confusion about this work. Therefore, we have changed $f$ in Def. 1 to $g$, so that it is not confused with a generator of an $f$ divergence. 

Finally, we also added references to two foundational books on submodular analysis so that readers unfamiliar with this field can more easily find an entry point.

**Competition with Deep Ensemble:** As said in the abstract and introduction, the aim of our work was to make deep ensembles more principled. We argue that theoretical contributions should be treated different from the empirical ones, please see the ICLR 2022 reviewer's guidelines: https://iclr.cc/Conferences/2022/ReviewerGuide (quoting):
* What is the goal of the paper? Is it to better address a known application or problem, draw attention to a new application or problem, or to introduce and/or explain a *new theoretical finding*? A *combination* of these? *Different objectives* will require *different considerations* as to potential value and impact.


**Related work:** Thank you, we will add the proposed reference to the related work. We note, however, that the goal of our work, as was the goal of Havasi et al., ICLR 2021 was to not correlate the ensemble members, but actually to *decorrelate* them.

**Comparison to f-POVI and Repulsive Ensembles:** Both of these works are, indeed, similar to ours. However, the major difference is that our optimization processes are very different. Neither of the papers train their models with BatchNorm and data augmentation, as these techniques lack Bayesian interpretation. It is thus unclear, what the performance of the repulsive ensembles is in practice, i.e. when one is using state-of-the-art training approaches. Our framework shows that 
1. there exists a Bayesian interpretation of greedy ensemble training,
2. and it works with state-of-the-art training techniques.

Previous papers failed to demonstrate the aforementioned points. To make it clear: POVI methods need to be compared with methods that do not use data augmentation and BatchNorm. 

Another point that we would like to make, is that the work on Repulsive Ensembles is not only in the same class, but also it has appeared **after** June 5th, which makes it too new to compare to according to the ICLR 2022 reviewer's guidelines: https://iclr.cc/Conferences/2022/ReviewerGuide.

Since this raised a concern on the reviewer's side, we have added a f-POVI related text to the Discussion section. 


 Dear all, 

Thanks for the provided feedback. We reply to all of you individually below. This post contains the summary of the changes made to the paper:

* We have made the connection between submodularity and ensembles clearer in the introduction.
* We have made the distinction between our approach and f-POVI clearer in the Introduction and Discussion sections.
* We have made the process of generating Figure 1 clear via updating the source code.
* We have fixed the abuse of notation in Definition 1.
* We have added relevant related work.

The revised version highlights changes from the first submitted version in red.

The paper presents a combinatorial approach to controlling and improving the effects of diversity; a relevant and interesting feature in Bayesian networks. The approach is general in the sense that it involves minimization of an f-divergence, for any f. The approach is mainly compared to standard deep ensembles, i.e. without controlling diversity. Additionally, the paper provides a theoretical result with approximation guarantees, which might be useful outside this specific application. PROS

1. The approach targets diversity, a relevant and interesting phenomena, and gives a technically non-trivial algorithm for controlling it. Especially, I like the connection to the empirical risk in Eq. (1) and the flexibility of the method (it works for any f-divergence). To me it seems the authors expertize in combinatorial optimization.

2. The results seem convincing that the proposed method can outperform standard deep ensembles in OOD tasks. Moreover, Figure 3 is a nice representation of the OOD detection results.

3. The Related Work section and the literature study in the introduction are dense. The authors have clearly spent time probing the field. These sections are nice to read and are useful for navigating in this evolving domain (although I miss one reference, see 2. below).


CONS

1. My main concern is regarding accessibility and clarity of how the method is described. As this is posed as a competing method/framework to standard deep ensembles, the paper should be accessible to practitioners and researchers with a broad technical background. As such, the concepts submodularity, supermodularity and modularity, as well as their definitions in the paper, need to be contextualized.

For instance, I have been working a lot with deep ensembles and Bayesian deep learning, and am interested in the method proposed in the paper, but I have a hard time understanding the motivation of using the modularities? I see that they help devise the method theoretically, but what are f, V, B and A in Definition 1? As an example of contextualization, simply state what f is in this setting. All I understand about f right now is that it is a mapping from a binary input of dimension V to the real line. Also, is f in Def. 1 the same f as in f-divergence? If not, please consider distinguishing them.

In my opinion, the delivery of the method in, especially, sections 2.2 and 3.1 is not easy to follow. These two sections essentially contain enumerations of definitions and I have a hard time finding motivations for them. As far as I am aware, you do not reference the definitions or the theorem in the text.

2. Although the sections Related Work and Introduction are thorough, I think [1] should be included. In [1] they correlate the ensemble components by letting them share weights. Your objective function also correlates the ensemble components. In fact, I think the results in the paper would benefit a lot from comparing with [1] *and*, e.g., [2]. [2] since, as is pointed out in Sec 5, it is "closely connected". Not comparing with POVI methods overall is not well motivated, in my opinion.

3. How did you obtain Figure 1? It is not obvious to me that (a) and (b) would occur. It seems speculative and unmotivated as is (this alone makes my "Correctness" score = 3).

4. The paper has an OK structure in the majority of the sections (especially Sec. 1, 5, 6 and 7) and the language is generally good. However, there are many disturbing typos which could easily have been caught by reading through the paper.

[1] "Training independent subnetworks for robust prediction", Havasi et al., ICLR 2021
[2] "Repulsive deep ensembles are bayesian.", Francesco D’Angelo and Vincent Fortuin. NeurIPS 2021 I am certain that the proposed method can indeed be useful and of interest for many in the field, and the method appears to be largely novel. However, I am concerned that it will not be very impactful, the way it is delivered now. To be clear, I don't believe that the mathematical approach is necessarily complicated. However, the provided explanations are over-complicated, and the approach is insufficiently motivated/contextualized.

To address this concern, I suggest that, for instance, redundant definitions (as Def. 4?) are moved to the Appendix (if at all needed). Instead, there should be some paragraphs in Section 2 that puts concepts like submodularity in context. Can you map it to a concept in Bayesian DL?

Regarding the baselines, I think it is important to compare the method to more recent advances in order to fully grasp its capabilities. As is shown in the Related Work, the field has evolved a lot since deep ensembles were first proposed. Question to the authors: how come more baselines were not considered?

This proposes studies ensemble construction from a combinatorial point of view. Analysis are conducted in function space, where the intuition are from and algorithm are developed. Specifically, it starts by formally defining the sub-modularity then proving f-divergence for a distribution and a mixture of kernels to be super-modular function. A greedy maximization of f-divergence is given and its implementation for approximating Bayesian posterior are presented. The experiments on synthetic data show the ability of covering the posterior of the proposed method, compared with the deep ensemble approach. It also conducts experiments on real-world data for OOD detection tasks.  In general, the paper provides a novel perspective for constructing the ensemble, connecting f-divergence between a distribution and a kernel mixture with sub-modular functions. Submodularity is useful in exploration, e.g., submodular set function maximization. The general idea is novel and intuitive, which could be helpful for the research in deep ensemble.

The execution of the implementation is solid in general. Some findings and analysis are interesting, e.g., a better approximation factor compared with theoretical guaranteed factor could be achieved for f-divergence. I have the following questions or comments before making the final decision:
1. Could you provide a high level explanation of why sub-modularity might be helpful for constructing the ensemble before introducing the definitions, theorems and algorithms? Jumping to this functional view after brief review of previous works might be too abrupt.
2. How are the mode-seeking and mean-seeking ability of the proposed method justified theoretically or empirically?
3. There are a few approximations like using the point estimate for the reverse KL. How does it affect the general performance and how is this quantified?
4. It is claimed the heuristic techniques like batchensemble or hyper parameter ensemble could be used adjunct with the proposed method. However, those methods are scalable to large datasets like ImageNet and could be computed efficiently. I wonder if the proposed method could be extended to ImageNet level datasets.
5. How is the performance of robustness under adversarial attack? This paper is with interesting idea and solid execution. I recommend a weak accept before the questions are answered.

The paper proposes a novel and principled method to make Deep Ensemble Bayesian. It minimizes an f-divergence between the true posterior and a kernel density estimator fitting the functions associated with the ensemble candidates. The authors formulate the learning as a submodular problem and propose to use a greedy approach to solve it. The resultant learning objective consists of a novel diversity term. Some out-of-distribution detection results in computer vision demonstrate the efficacy of the proposed method. # Strengths
- The problem of making Deep Ensemble Bayesian is important for the community, and the proposed method seems to be a viable approach to solve this problem.

- To my knowledge, the idea of rephrasing the Bayesian inference problem as a submodular one is conceptually novel.

- The theoretical justification is impressive and interesting.

# Weaknesses & Concerns

- It seems that, eventually, you use the reverse KL divergence as an instantiation of the f-divergence, and develop the practical training loss Eq 12 and Eq 14. 
Though your modeling is applicable to arbitrary f-divergence, you have not demonstrated the implications of such universality.

- Eq 14 is closely related to the function-space POVI (Wang et al., 2019). The difference is that you train these ensemble candidates sequentially with weight decay while function-space POVI trains them in parallel without weight decay? I would like to see an empirical/theoretical clarification of the connections/difference between these two approaches.

- Although I know the main contribution of this work is on a new theoretical framework for Bayesian Deep Ensemble, I still want to see that this works can beat Deep Ensemble significantly in more scenarios. The current performance gains in Table 1 are marginal. This paper has done a great job in making Deep Ensemble Bayesian theoretically, despite being short of empirical verification. So, currently,  I recommend a weak acceptance for it.

Meta-review: The authors present a new framework to make deep ensembles provide better coverage of the posterior and be less reliant on initialisation. The authors generally did a good job presenting their approach, avoiding dubious claims that deep-ensembles are non-Bayesian, and instead focusing on ways in which deep ensembles can be improved, practically and theoretically. It is worth noting in a revised version, however, that many approximate inference procedures do not have theoretical guarantees. The claim that deep ensembles have "arbitrary bad approximation guarantees" is vague and appears to single them out in a way that could confuse the reader. Regarding priors, it is also worth noting that Wilson & Izmailov (2020) provide evidence that the prior in weight space induces a prior in function space with useful properties, although the prior can be improved.

The authors do a good job of responding to reviewers, and describing limitations. Ultimately, however, the general opinion was not swayed to accept. In addition to reviewer concerns, the experimental evaluation could be substantially improved. There are several procedures that build on deep ensembles to capture uncertainty within modes. How does this procedure compare? Why are no likelihood evaluations considered? What about accuracy? In its present form, it's unclear what practical value the contributions are providing, besides possibly better OOD detection, but even that direction is explored in a relatively limited way. It could also be interesting to measure the distance of the predictive distribution to a good proxy for the Bayesian model average. Overall, there are the raw ingredients of a good paper here, and the authors are encouraged to continue with this work.

Decision: Reject