Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper considers learning a meta-model to predict the solution to the OT problem.
strengths_and_weaknesses: The idea is novel and the paper is well-written.

Compared to other OT methods that run from scratch, it can save the time of learning OT potential for a new task. However, the experiments in this paper are only 2 or 3 dimensions. And I don't see how it can scale to a high dimension in the current framework (figure 3). For example, if $\alpha,\beta$ are a dataset of images and each image is a sample point, then $z_1$ and $z_2$ would be of very high dimension, and I conjecture MLP will not work anymore. And if it's only this kind of low dimension, for example, do people really care about the difference between 1e-5 seconds and 1e-3 seconds? I have a question mark about its real application.

Also, the pipeline in Figure 3 is quite problem-oriented. Given a new format of distributions $\alpha,\beta$ that are not images, for example, single-cell RNA data or a set of point clouds, one needs to re-design or re-select the encoder of $\alpha,\beta$. ResNet may not be able to extract the correct information.

Regarding the continuous measure $W_2$ solver, I think W2GN is not a good choice, at least not the best. It primarily considers OT maps in latent space of pre-trained autoencoder, which limits practical application in high dimensional space, see the discussion in the second paragraph of [Rout et al. 2021](https://openreview.net/pdf?id=5JdLZg346Lw). The best continuous $W_2$ solver has been shown to be MM:R in [W2 benchmark](https://openreview.net/pdf?id=CI0T_3l-n1) paper. Why do authors choose to use outdated W2GN instead of state-of-art $W_2$ solver?

Since the computation speed is a highlighted advantage, the authors should also consider the comparison with the GeomLoss package (https://www.kernel-operations.io/geomloss) and “Fast geometric learning with symbolic matrices” Feydy et al., NeurIPS 2020. It's a GPU implementation and it supports arbitrary cost functions and scales up to millions of high-dimensional samples in seconds.

The authors can be more careful about the literature:

Line 235: Li et al. 2020 didn't use ICNN. Instead, I find these two papers use ICNN

Alvarez-Melis, David, Yair Schiff, and Youssef Mroueh. "Optimizing functionals on the space of probabilities with input convex neural networks." Transactions on Machine Learning Research 2022.

Fan, Jiaojiao, Amirhossein Taghvaei, and Yongxin Chen. "Scalable computations of Wasserstein barycenter via input convex neural networks." ICML 2021.


questions: I have a question about the line 49-50 out-of-sample setting. If Meta OT can approximate the potential of W2GN well, then it should be able to apply to any samples in the considered task, not only just training samples.
limitations: see weakness
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this work, the authors introduce a new and very efficient procedure to solve optimal transport problems between both discrete and continuous distributions using amortized optimization. More precisely, in the setting where one aims at solving multiple OT problems between distributions sharing similar structures, the authors propose to learn, in an unsupervised manner, a predictor able to infer the optimal coupling instead of solving from scratch each OT problem considered. Such learned function can then be applied to quickly predict the optimal coupling between two test distributions (in the sense that they have not been used in the training stage) sharing the same structures as the distributions considered in the training stage. To do so, the authors propose to learn a function parametrized by neural networks mapping the input distributions and the underlying cost to the optimal potential. In order to learn such function, the authors  minimize the expectation on a distribution of problems (defined by two input measures and an underlying cost) of the dual objective of the OT problem under the same constraints. Note that if the family of functions generated by the network is sufficiently expressive, then solving such problem is the same as solving each single OT problem and therefore finding the true optimal potential on each of these problems. Note also that the proposed method is completely unsupervised as they do not require to solve each OT problem considered in the training stage. In the discrete setting, the authors consider the dual formulation of the entropic OT which is an unconstrained problem, therefore they end up with an unconstrained optimization problem studied and propose a simple gradient-descent procedure to solve it. In the continuous setting, the authors restrict themselves to the learning of the Wasserstein-2 distance, and take advantage of a reformulation of its dual involving convex potential to recover an unconstrained optimization problem too. Indeed in this case, the authors aim at predicting an ICNN by learning a function mapping the input measures to the parameters of the ICNN. The authors propose also to refine the prediction on test problems by applying few steps of either the Sinkhorn algorithm in the discrete setting or of the W2GN algorithm in the continuous one. Finally, they show on various experiments on real-world data that the proposed approach is able to recover almost the true potentials (and couplings) while being much faster. 

strengths_and_weaknesses: I think that the paper is of very good quality, very clear and concise, and that the contributions presented here may be of real interests for practitioners using OT. More precisely, I think that learning a map that given a geometry is able to infer the optimal coupling for a family of problems is a very promising line of work which may be useful in various setting. For example one could think at some networks which during the forward pass, aim at some point to align data and therefore for each forward pass, one OT problem has to be solved. Another example could be that some studies require to compute multiple OT problems between similar point clouds which can be very time-consuming. The approach proposed here could significantly improves the computational time of this kind of tasks. I think that the authors should even motivate more their contributions by providing some contexts where the proposed learning procedure could be useful. 
questions: How do you take into account that the learned dual potential is also a function of the underlying cost in the discrete setting? As it seems that the MLP takes only into account the atoms of the measures.

How does this method compare to the supervised approach where we consider the exact same parametrization of the potential however we aim at minimizing the distance between the predicted potential and the true one?


How long is the training stage? 
It would be nice to provide for each experiment, the plot of the training loss againts the CPU/GPU time or the number of operations. It would be nice also to see how versatile can be the model in the sense that if the network considered is sufficiently large, then it may be able to learn the optimal potential in various settings and not only for a specific distribution of problems.

limitations: Yes.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: As an experimental paper, Meta OT provides better initialization to OT solvers by leveraging shared representations from previous tasks. It helps accelerate the training of OT solvers which is well supported by experimental results provided in this paper. One key requirement is that downstream tasks must follow the identical distribution as the previous tasks. This was acknowledged by the authors under the limitations of meta OT.

In general, the paper is concise and well-written. The main idea is clear. My major concerns are listed in the main review. I am willing to raise my score if these concerns are properly addressed. 


strengths_and_weaknesses: ## Main Review
### Strengths
1. The idea of predicting the parameters of convex Brenier potentials approximated by ICNNs is a major strength due to the following reasons. First, directly learning the parameters of an ICNN while satisfying the convexity constraint is shown to be challenging in prior works. Second, the gradient of the ICNN may not contain the true OT map due to its poor expressive power. This paper takes a step towards resolving the first challenge by learning to predict its parameters, e.g. through a ResNet and MLP. As an extension, one might increase the complexity of ICNNs which may resolve the second challenge.
2. While prior works have tried to directly approximate the gradient of Brenier potentials using a neural network, there is insufficient theoretical evidence on whether the learned neural network is indeed an OT map. On the other hand, this paper uses a meta-learning approach to directly learn the parameters of Brenier potentials. Thereby, it ensures the optimality of the learned transport map as per Brenier’s theorem.  

### Paper organization/presentation
1. The paper is well organized and nicely written. 
2. The presentation is clear and concise. 

### Experiments
1. At present, the experiments are conducted on relatively easier datasets. 
2. The lack of quantitative results makes it hard to evaluate the overall performance. 
3. Ablation study is needed to measure the distance of fully trained weights from random initialization and Meta OT predicted initialization. Whether a better initialization would give results comparable to Meta OT?

### References
1. The paper includes a comprehensive list of relevant literature. 
2. One suggestion would be to cite the published version of papers where applicable. 


questions: Please see **Main Review** and **Weaknesses** for questions to address during the rebuttal. 
limitations: ### Weaknesses:
1. In checklist 2, the authors mention that the paper is not a theory paper. In my opinion, the paper currently lacks sufficient experimentation that may vouch for its clear acceptance. I suggest the authors include stronger experimental results on CIFAR10, CelebA, CelebA-HQ, or other relevant datasets. If time permits, the authors may choose the task of generative modeling or image restoration where OT has shown promising results. How efficiently can Meta OT predict the parameters of the OT map in the aforementioned tasks on these harder datasets?
2. The paper claims that we would receive a stream of new images in deployment which could be different from the images used to obtain the OT map (lines 31-32). However, the individual experiments on MNIST, Spherical, and WikiArt are conducted with samples from the same distribution. I understand that while standard OT solvers need retraining from scratch, Meta OT provides a better initialization. The authors should discuss to what extent Meta OT can handle the stream of new images in deployment.
3. It would be helpful to assess the performance of color transfer if the authors provide quantitative results. It is hard to judge qualitatively since images visually look all the same. I suggest both perception and distortion metrics to analyze perceptual quality and geometric distortion in the pushforward samples.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper views the dual problem of optimal transport via the lens of amortized optimization. Then the authors propose a novel method to efficiently predict the optimal transport maps from the input measures. The predicted solutions can be then used as an initialization for standard OT solvers. Empirically, this method improves the computational time of standard OT solvers by multiple orders of magnitude in both discrete and continuous settings.

strengths_and_weaknesses: While the use of amortization in OT is not completely new, this work proposed a novel approach to find good initializations to solve multiple OT problems more efficiently. Experimental results back the computational advantages of Meta OT. Using the initial prediction from Meta OT, standard OT solvers converge much faster. However, there is no demonstration of some standard machine learning tasks that famously gain benefits from OT (e.g. generative modeling, domain adaptation). Therefore, it is difficult to judge the “quality” of transport maps of the proposed solvers in practical applications. Overall, this paper is well-structured and easy to follow. There are some minor parts that can be improved to increase the readability of the paper. 
questions: I have the following questions:
1. The sentence in L158 is not clear. Could you explain it in more detail?
2. What is the formula for the marginal error in Table 1 and Figure 4? Is it Equation (7)? What if we use a stricter threshold like 10e-4, 10e-5, etc. Does MetaOT + Sinkhorn still converge faster? 
3. What is the formula for the normalized dual objective value in Figure 7? Is 1.0 the optimized value?
4. **Related work.** I came across a recently published paper “Rethinking Initialization of the Sinkhorn Algorithm”. Because both papers try to generate an efficient initialization for speeding up the Sinkhorn algorithm, it may be useful to cite and discuss that paper.

**Minors:**
* *Sinkhorn algorithm.* The update for $g_i$ in Algorithm 1 should use $f_i$ instead of $f_{i-1}$.

limitations: The authors provided two limitations of Meta OT which may be practically important but are out of their problem settings. Other than those two, it would be interesting to see the applications of Meta OT on deep learning tasks such as generative modeling or domain adaptation.
review_title: null

Meta-review: metareview: This paper proposes an amortized optimization approach for predicting optimal transport (OT) maps. Three reviewers found that the proposed method is interesting. However, some concerns raised by another reviewer on the improvement of the computational efficiency and the generality of the proposed method were raised: 

1) The experiments on computational efficiency are of very small scale, and insufficient to justify the improvement. The authors may consider larger scale problems, where the run time should be significantly larger than other computational overheads, e.g. >10 seconds.

2) As shown in Tables 1 and 2, the improvement of the proposed method on computational efficiency is marginal.

3) The current experiments only use MLP for small scale and relatively easy low dimensional problem. To demonstrate the generality of the proposed method, the authors should consider other neural architectures for more complex data.

This paper can be significantly strengthened if these issue could be addressed.
confidence: Less certain
Decision: Reject