Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper studies why and when averaging the weights of a model improve out-of-domain generalization by way of a new a bias-variance-covariance-locality decomposition of the expected target domain error. They show how diversity is needed and thus propose Diverse Weight Averaging (DiWA) wherein weights of N independent trained models from shared initialization are averaged together (in an optionally greedy fashion). They show how DiWA outperforms weight averaging across N model checkpoints of a single training run and other baselines on the public DomainBed benchmark.

strengths_and_weaknesses: Strengths:
* The paper is very well written -- exposition is clear, well-organized, high quality (free of typos), and each section is well-motivated. For example, "Limitations of the flatness-based analysis" discusses why the current explanation for weight averaging's performance is insufficient and why a different type of analysis (like their 4 term decomposition) is needed.
* The proposed bias-variance-covariance-locality decomposition (Proposition 1) is novel and aids our understanding of weight averaging. Furthermore, it directly motivates their proposed DiWA.
* DiWA is evaluated on DomainBed, which is the standard public benchmark for out-of-domain generalization right now. Experimental setup is sound and the comparisons with baselines fair.

Weaknesses:
* Despite DiWA outperforming weight averaging checkpoints of a single run (WA), it is unlikely to be adopted for some use cases. For example, for large language models, there is no extra cost for doing WA but training the model multiple times as needed by DiWA is not practical or feasible. The extra compute is probably better spent scaling the model size or training for longer.
questions: I would be curious to know how DiWA and WA would fare when the Sharpness-Aware Minimization (SAM) is used in each training run. Perhaps, the benefits of DiWA or WA would be diminish in the presence of SAM?

limitations: Yes.
* "extreme hyperparameter ranges lead to weights whose average may perform poorly"
* "diversity is key as long as the weights remain averageable" -- linear connectivity in weight space is needed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors proposed a simple method to improve the out-of-domain generalization, i.e., by averaging weights from different training runs rather than one run. Some theoretical analysis is also given for explaining weight averaging (WA) and the proposed method. Experiments on the DomainBed benchmark show the performance of the proposed method. 
strengths_and_weaknesses: Strengths:
1. The research question of when WA succeeds and how to improve it, is interesting and relatively less explored.
2. The paper provides theoretical insights for WA and the proposed method.

Weaknesses:
1. The writings of the paper may have some overclaim issues. In the abstract, the authors claim that "for out-of-distribution generalization in computer vision, the best current approach averages the weights along a training run" (line 2-3). But from the introduction, it seems that such a claim is only based on the observation from the DomainBed benchmark (line 22-24). I'm wondering does the observation on one benchmark could represent the whole situation in computer vision? 
2. The contribution of this paper is quite minimal. Especially regarding the experimental results, it seems there is no large performance improvement of the proposed method when compared with MA [29] in Table 1; under the random initialization, the performance of the proposed method looks similar to the performance of SWAD [14].   
3. The proposed method needs to ensemble weights from different runs (e.g., >=20 runs in the experiments), which may cause unnecessary additional computational cost.
questions: 1. How to decide the total number of runs? Is the number larger usually mean the final performance is better?  
2. Is there any intuition of why using MMD in Proposition 3? How about other divergences?
limitations: The authors have addressed the limitations of their proposed method.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes Diverse Weight Averaging (DiWA), which averages weights obtained from different training runs starting from the same initial parameters and mildly different hyperparameters. The paper decomposes the expected OOD error into (bias, variance, covariance, locality) and claims that weight averaging is most effective when the covariance term dominates, verified empirically. They show improved OOD generalization performance on the DomainBed benchmark.
strengths_and_weaknesses: Overall, I think this is a solid paper with a conceptually simple method and clean writing.

Over ENS, WA has the benefit of requiring only one feedforward at test time. Both Lemma 1 and figure 1 show that WA is a close approximation to ENS in the training setting of DiWA. In fact, WA seems to perform slightly better than ENS.

DiWA is simple and effective, and the experiments show that it can be combined with previous advances for more benefits: LP initialization, leveraging diversity from ERM/Mixup/Coral algorithms, etc.
questions: What is the performance of ENS using the models obtained by DiWA?

What is the rough training cost of obtaining 60 models for the final results? Were you unable to obtain error bounds in table 1 because of the computation costs?

Minor comments
Line 89: “…why WA outperforms in OOD Sharpness-Aware Minimizer…” I’m unsure whether the grammar here is technically wrong, but I took a while to parse this sentence.
limitations: To my knowledge, this work has no potential negative societal impact other than what was already present in the existing literature.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors propose an approach to mitigating the generalization problem in computer vision, under the common scenarios of either a difference in the data generating marginals (covariate shift) or a difference in the class-conditional distributions (which they call concept shift or correlation shift).  They connect previous work on the errors incurred by weight averaging and standard ensembling, and offer a new decomposition that better explains when weight averaging fails under correlation shift or covariate shift.  Their proposed method of averaging diverse sets of weights leads to improved results on the DomainBed benchmark suite of datasets.  

strengths_and_weaknesses: ### Strengths:

The paper is well written.  The sections are subdivided cleanly and they preview the arguments established in each subsection, culminating in a larger point.  The authors are clearly well versed in the literature of ensembling and transfer learning.  

In particular, section 2.4 and its subsections develop the argument for why weight averaged ensembles are robust to covariate shift (or diversity shift) quite convincingly.  

Also, the analysis in section 2.2 of why flatness-based minimizers do not have any effect on OOD error was illuminating.

### Weaknesses:

There are a few small issues that I'd like to see addressed:

At the outset in the abstract, the authors state that the best current approaches for out of distribution generalization are derived from averaging the weights along a training run.  This is not quite so, as there are multiple other approaches (SWAG, Deep Ensembles, LNNS, Loss surface simplexes for mode connecting volumes and fast ensembling). These all introduce greater member diversity though different mechanisms, so it would be more correct to broaden the related work section to include them, and to acknowledge them in the abstract.

On lines 185-186 the authors state that hey follow the objective of decorrelating the learning procedures of ensemble members, citing the DICE paper.  It’s an exaggeration to say that the DICE objective is followed in section 3.  The authors of DICE perform considerable work to implement a variational approximation to the conditional entropy bottleneck between two learners.  The authors of this work do not go to such lengths to decorrelate the features extracted from the inputs by ensemble members.

### Comments:

I think the paper would benefit from a perspective that comments upon the difficulty of transfer learning based on the presence of neural collapse, or learning neural network subspaces, especially as the authors note that the size of the space spanned by the furthest member from the barycentre is important.

It's hard to believe that this paper's claims of increased diversity by new initializations are an actual advance, this is present in all the modern ensembling literature (cf. papers descended from the DeepEnsembles literature).  In Table 1,  there does not seem to be a comparison against simple deep ensembles ($f_{ENS}$ in the notation of the paper.)

A question I have with the setup in 2.1 is why would we not want to compare against gains made by fine-tuning in the target distribution.  In most practical scenarios, there is limited target data available to train on.  Certainly in almost all modern NLP tasks, this is a baseline to consider.  Including this baseline, where a model (or ensemble of models) that is trained on S is allowed to be fine-tuned in increments (of data or learning iterations) on data drawn from T before being evaluated, would qualify the degree to which DiWA is a feasible solution to practical transfer learning.

Overall, despite some small quibbles, I find that there is a lot to offer in this paper and the community would benefit from seeing it in a first class venue.

questions: One thing I found inconsistent, was the combination of the results of Lemma 1 and the evidence in Figure 1.  Lemma 1 relates the loss of a weight-averaged model and a prediction-averaged model, showing  that the loss of the weight averaged model is the same as the prediction averaged model, plus a (non-negative) term based on the max of the 2-norm between the weight average parameters and an individual member.  Yet if that is so, I would expect prediction averaged models to outperform weight-averaged ones.  Figure 1 establishes that in fact the opposite is true.  Is it the case that I am misinterpreting Lemma 1?

An answer might lie at the end of section 4 on lines 256-266, as well as part of Appendix D: "These experiments confirm that diversity is key as long as the weights remain averageable.".  So, in effect, weight averaging performs well when the members being averaged lie within  a low-error subspace that is compact.  I would like to see this concept of "averagable" explored further.  There seems to be a practical friction between training diverse members of the ensemble (say by varying the initializations) and ensuring that the members do not stray too far apart so as to no longer stay averagable.  A closer investigation of when this happens (say from Figure 5) in terms of violations of the linear mode connectivity would be illuminating.
limitations: Yes, they have.
review_title: null

Meta-review: metareview: The reviewers unanimously recommend accepting the paper - congratulations!

My only concern is in the related work: The submission mentions

> The recent “Model soups” by Wortsman et al. [28] developed a WA algorithm similar to Algorithm 1. However the task, the theoretical analysis and most importantly the goals of these two works are different.

This is not an accurate characterization because Wortsman et al. [28] were also interested in out-of-distribution generalization - their paper mentions "robustness" and "distribution shift" several times and contains results on multiple OOD test sets. The results in this submission and in Wortsman et al. [28] reinforce each other since the two papers evaluate on different OOD benchmarks and find that weight averaging helps in both. I encourage the authors to clarify this in their related work section so that the reader can correctly put the results in context.
confidence: Certain
Decision: Accept