Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The article proposes an iterative procedure to compute Wasserstein barycenters, the parametrised barycenter is modelled as a latent distribution pushed forward by a neural network. 

The authors study the relationship with existing methods and use their proposal to build a benchmark dataset for barycentric computation
strengths_and_weaknesses: The paper does a great job in the literature review, where it puts its method in context and compares it with other techniques in terms of their advantages and disadvantages.

The theoretical treatment of the contribution is well explained, the algorithm consists in iteratively applying a neural operator to a (latent) density according the known fixed point barycentric method by [3]. The proposal is validated in the celebrity dataset. 

In my opinion, the weakest point of the article is the experimental validation. Though the proposal is sound, only two sets of experiments are shown: i) evaluation of a synthetic dataset built upon the proposed method, and ii) other synthetically-generated dataset. These experiments hardly validate the idea of a general method for barycenter computation, since there is no evidence that the proposed method operates beyond datasets built in the specific manner mentioned by the authors. This issue is identified in the manuscript, in Sec 7 (lines >295) where the authors acknowledge that their dataset only cater for specific (dispersion) properties - given this observation, it was clear that the experimental validation should've included richer datasets. 

This weakness in the contribution is further clarified when the authors identify a possible application in MRI (Sec 7), where datasets have limited overlap. Perhaps a stronger experimental validation could include real world datasets of that kind.

Though well written in general, the article could benefit from a proofreader 
questions: How can the author justify the general applicability of their method given that the experiments only consider purposed-built datasets? 
limitations: the limitations of this work are identified in the manuscript
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Authors propose a methodology for estimation of wasserstein barycenter in continuous setup. Authors argue that it beats the state of the art. In the passing authors conceive a new way of transforming/splitting datasets leading to new 'ave, celeba' dataset which might be useful in the future for benchmarking in OT tasks.
strengths_and_weaknesses: The main strengths are that it is a thoughtful paper on a relevant topic that keeps demanding new methods, and where the proposed methodology is valuable because the 'tricks' are novel, and because they lead to better results. It is clear to me that there is a large amount of work involved on this project.

The new dataset is also a major strength of this paper, and I hope authors will work in order to make it easily publicly available for future work. Although the math is rather simple, it is a strength of the paper that the use of math leads to quite valuable methodological tricks.

The main weakness is that although there are certainly quite a few clever methodological nuggets, altogether it is very hard to identify how everything is put together to lead to a better algorithm (see for example questions below). If that is not clear enough then the community has reasons to be skeptical about the evaluation. This is mostly an interpretability issue and I hope it can be addressed.

This requires some work in two related aspects: 1) although writing is clear, what is less clear to me are the different components of the proposed algorithm.  Figure 2 is beautiful but it is not clear enough. Here I missed the usual Algorithm box. I believe authors should try to polish the methods section (4) so it is crystal clear what the contributions are. I would have love to have a more thorough discussion than 'it works better empirically'. Hopefully, we would see a clear justification to every design choice made in section 4.
questions: Why do you only compare against SCW2B? I understand that this is the state of the art but by only comparing against it, and given that there are multiple components to your methodology, it is hard to identify where the gains come from. I suggest to whenever further experiment that allow us to debug where the gains come from. What are the advantages to each of the components in section 4?

I strongly suggests to add a toy example. Is there any reason for why this should be not necessary? For example, one could try to find the barycenter of some simple continuous gaussian data. Adding such toy example (which suitable baselines, even if they don't correspond to the SOTA SCW2B) would help 'debug' and interpret the computational gains.

Methods related: on many parts it is mentioned that ICNN don't work so well but at the same time the congruent functions are computed with ICNN. Could you make this distinction more explicit in the text (why for dataset creation it makes sense to use ICNN while they don't work so well for barycenter estimation?)
limitations: To me understanding authors correctly discuss limitations. But lack of sufficient interpretability is a substantial limitation of this work, as it is submitted at this point.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Computing the Wasserstein barycenter between continuous measures is an important problem. In this paper, authors propose a method based on a fixed-point approach to compute the Wasserstein barycenter with arbitrary neural networks. Moreover, they introduce a way to create datasets to compare to quality of the obtained barycenter and compare their method to a ICNN based method on the "ave, celeba!" dataset, which they generated using their method.
strengths_and_weaknesses: 
This paper propose to use generative model into the fixed-point approach previously introduced. This allows to use NNs to parametrize the barycenter and to obtain better results than other papers. Moreover, they introduce a way to construct benchmark datasets to compare the efficiency of the method to compute Wasserstein barycenters. This is a very nice contribution which can be very helpful to compare methods.

Strengths:
- Fixed-Point approach with neural networks and the regression approach.
- A new way to construct a benchmark dataset to compare methods on Wasserstein barycenters


Weaknesses:
- Comparison with only one method. A more complete comparison with the different methods on the newly introduced dataset could be nice.
- Still some open questions on whether the fixed-point approach approximates the barycenter.

questions: Some related works are not cited. In [1], a method is proposed to compute continuous Wasserstein barycenter for abitrary costs. I believe it would be worth also comparing the method against their method.

[2,3] are also a missing reference on the methods used to approximate Wasserstein barycenters.

The method seems pretty heavily computationally (since it requires to compute N OT maps at each iteration and solve a regression problem), but I guess it is also the case of the other method which is compared with. I would be curious to see some comparison of the training time between the methods.


[1] Chi, Jinjin, et al. "Variational Wasserstein Barycenters with c-Cyclical Monotonicity." arXiv preprint arXiv:2110.11707 (2021).

[2] Backhoff-Veraguas, Julio, et al. "Stochastic gradient descent in Wasserstein space." arXiv preprint arXiv:2201.04232 (2022).

[3] Daaloul, Chiheb, et al. "Sampling from the wasserstein barycenter." arXiv preprint arXiv:2105.01706 (2021).
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes to learn the Wasserstein barycenter through the fixed point iteration method. They interchangeably 1) solve the OT map between the barycenter candidate and each marginal distribution, 2) update the barycenter candidate by fixed-point iteration, then regress a generator to fit the barycenter candidate. It also develops a CelebA barycenter dataset that has a known barycenter solution.
strengths_and_weaknesses: Strengths:

It provides a nice interpolation between the gradient updates of the least square regression to fixed point barycenter update and the variational barycenter algorithms.

It gives theoretical guidance to generating probability measures with known Wasserstein barycenter.

It obtains advantageous performance compared to the competitive SCWB algorithm.

Weaknesses:

As at the beginning of section 4.1, the fixed point doesn't promise to be the barycenter, and the entire sequence may not converge, which may cause numerical issues. These two challenges would question the foundation of the proposed fixed-point iteration solver.

The paper claims to tackle challenge c, but the algorithm still needs to compute N maps. So it doesn't decrease the calculation burden, at least when compared with SCWB.

It seems to me that computational time is rarely mentioned in the paper. For example, the paper uses $k_G=50$, and also a large number of iterations $k_v=50$, while the corresponding parameter is only 1 and 10 for SCWB. I conjecture the computational time of this method is several times larger than SCWB.
questions: The paper claims they cannot obtain the inverse map: marginal distribution -> barycenter during training, but the inverse map can be recovered by $x- \nabla v(x)$ if using eq 7 to solve the optimal map. This is discussed at the end of section 2 in [W2 solver benchmark paper](https://openreview.net/pdf?id=CI0T_3l-n1). Why not use this inverse map as a baseline?

In Figure 5, SCWB is particularly bad for the map corresponding to $P_3$. This is somehow weird. Do authors have clue why this happens?

For the ablation study, I think it would be helpful if the authors can add some experiments with $k_G=1$. Thus we can be more clear about whether SCWB has worse performance because ICNN is not expressive enough.
limitations: see weakness
review_title: null

Meta-review: metareview: This paper proposes a new iterative method for Wasserstein barycenter based on a generator parametrization of the barycenter and fixed point method that alternates learning the generator and learning  the OT maps from barycenter to measures. The work is empirical and lacks theory but proposes a new image benchmark on celeba for barycenter evaluation. Reviewers were positive about the paper. Accept.  
confidence: Certain
Decision: Accept