Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors first take a generalization to the strong and weak optimal transport problems by replacing the objective function from an expected cost wrt the transport plan to a general lower-semicontinuous functional. This formulation coincides with the regularized optimal transport, however, it is easier to adjust under different domains as one can directly design the functional rather than the additive regularizer.  It is shown that this problem admits a strong dual which is a max-min optimization problem, resembling the Kantorovich dual. The authors then show how to retrieve the optimal transportation plan from this dual, which coincides with the saddle points. By exploiting this result, they derive an NN-based solution algorithm, and afterward, they show how one can benefit from such a framework in a case where the underlying domain is concerned with having a "cost" representation that is beyond distances. They illustrate the performance of these methods over various numerical experiment settings.
strength_and_weaknesses: **Strength:** Firstly, I would like to thank the author for such a clear motivation, literature review, theoretical analysis, numerical experiments, implementation, and discussions. I am **not** an expert in OT, but I used it in my research a couple of times, and as far as I am concerned the paper looks like a great fit for the community.

**Weaknesses:** 
*I list some minor weaknesses. I am going to stay active during the discussion period in case the authors update the document or have questions about my review.*

***Minor Weaknesses/Questions***
- Section 1 starts very 'quick'. What is "unpaired restoration and style translation", for example?
- Section 2: In the beginning we have $\mathcal{X} = \mathcal{Y}$. Is this a typo?
- For Wasserstein-1 and Wasserstein-2 terminologies in Section 2, the given citations are specifically recent NN-based papers. However, the terminology definition comes from much earlier if I am not wrong.
- "General OT" (5) requires citations. It may look like this is a new definition.
- Section 3.1: "which alternative to (6)" -> typo
- Theorem 1: "minimax" -> maximin ? (The rest says maximin)
- Question: Is (11) used in any other paper before? If not, Proposition 1 can be a theorem.
- Section 6: "as well as the trained networks" -> "along with" is maybe better?
- Section 3.3 looks shorter than it has to be, in my view. There is a huge relevant literature in Stochastic Programming, but here it is a little hard to relate. The proposed unbiased estimator comes without discussion. 

**UPDATE**
After reading (Korotin et al., 2022c) I am less enthusiastic about this work. I like the paper, but in the presence of the other one, I think the setting taken here can be derived relatively easily. I would like to update my score to "5" instead of "8". 
clarity,_quality,_novelty_and_reproducibility: For the details of these points, please see my review of "Strength and Weakness". However, to give a high-level answer:

**Clarity:** The paper is written very clearly. It is easy to follow. The theoretical contributions are clearly stated and the limitations are honestly discussed.

**Quality:** The paper has high quality. The experiments, visualizations, and results are thorough. The proofs look correct to me.

**Novelty:** The paper has novelty, and can guide the community toward a very modern adaptation of OT thanks to Section 4.

**Reproducibility:** The paper provides all the source codes, and the main algorithm is very clear. I cannot foresee any issues with reproducibility. The limitations are also discussed.
summary_of_the_review: This is one of the best papers I have reviewed in a long time. From the accuracy of the literature review to the solid theoretical results, useful reformulations, interesting experiments, and honest but very efficient solution algorithms, the paper is written extremely well.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper is addressing the problem of finding optimal transport plans for general cost functionals. The algorithms is based on a saddle point reformulation of the problem where the marginal constraint is formulated as maximization. The plan on (x,y) is represented as a stochastic map from (x,z) to  (x,T(z,x)) where z is the latent random variable and T(z,x) represents the conditional distribution p_{y|x}, in a similar style as conditional GANs. The application of the proposed algorithm to a data transfer problem with class preservation is described and the results are illustrated. 
strength_and_weaknesses: Strength: 
- the general ideas considered in the paper are novel and interesting. 
- the application that considered in the paper is interesting. 
- the mathematical results seem correct

Weakness: 
- the discussion of related works does not go deep and it is merely a list. 
- there are many results presented that makes reading the paper a bit difficult. It might not be necessarily relevant to the main objectives. I think it is better to reduce them and expand the main ones, adding explanations and clarifications. 



clarity,_quality,_novelty_and_reproducibility: The language of the paper is clear and numerical experiments seem reproducible. 
summary_of_the_review: I think the paper has enough content to be a good contribution but It should be presented in a way that highlights the main contributions and a more deeper discussion of the related works.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This present a novel neural-networks-based algorithm to compute optimal transport (OT) plans and maps for general cost functionals. 
The algorithm generalizes prior OT methods for weak and strong cost functionals. 
And it constructs a functional to map data distributions with preserving the class-wise structure of data.
strength_and_weaknesses: Strength: A functional is constructed to map data distributions with preserving the class-wise structure of data.

Weakness: the general cost functionals lacks designed and selected in detail. 
Learned stochastic OT map via the algorithm that may be fake solution. 
In Image-to-Image Translation task, Lack of comparison with other neural network-based algorithms for solving OT mappings, such as using Input Convex Neural Networks to estimate OT mapping.
clarity,_quality,_novelty_and_reproducibility: The overall structure of this paper is clear and the quality is average. 
In particular, the key of this paper is add class information to neural optimal transport, lack novelty.
The experiments can Reproducibility.
summary_of_the_review: Although the overall structure of this paper is clear, it lacks novelty. The experimental results do not show the superiority of the algorithm, and it is difficult to design a specific task cost function, lack of guidance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors proposed to estimate optimal transport maps that are parameterized as neural networks.
The proposed method is established based on reformulating the Kantorovich dual problem as maxmin problem which optimizes for a stochastic transport map and a potential jointly. Then, the authors also introduce a cost metric that encourages the transport map to preserve the class information, when mapping two datasets with conditional labels. 
The authors evaluated their proposed method on a few datasets, including toy synthetic data and real-world image datasets. The proposed method is shown to be effective. 
strength_and_weaknesses: Pro:
-This manuscript is pretty well-written and easy to follow. 
-The proposed method is well motivated as the authors comprehensively introduced different OT problem settings step by step.
-The experimental results are well-organized. The authors provided both qualitative and quantitative results to evaluate their proposed method.

Cons and questions:
-There might be some concerns regarding the novelty. The proposed objective, as well as algorithm 1 are similar to that in one of the cited works [Korotin 2022c]. Although in this paper, the authors computed a class-guided cost.
-In equation (11) and Proposition 1. It seems that the stochastic map is asked to map one mixture towards another. However, how to define the correspondence of classes between different domains? What will happen if the number of classes is different between two datasets? This may break down to the optimal transport between two mixture distributions. 

clarity,_quality,_novelty_and_reproducibility: This paper overall clearly demonstrated an approach to estimate optimal transport with neural networks. I would expect the authors to further elaborate on the contribution and distinguish this work from previous ones since the weak-Cost formulation and algorithm 1 are very similar to [Korotion 2022c]. The clarity is mostly good. 

summary_of_the_review: Overall this paper well-presented. Although the proposed method seems to be a little bit incremental since it is built upon existing formulation and algorithms. Since the presentation quality is high, I lean towards acceptance but I am still on the fence.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a framework and an algorithm for solving a so-called general OT problem.  

While reviewers agree that the paper provides a solution to a new problem, after the live discussion, we agreed that the contributions are incremental compared to Korotin 2022c and other (strongly related) papers submitted to ICLR, especially from the theoretical perspective. When we look at the practical contributions (on class-guided NOT for instance), we also concur on the fact that the addressed problem (with partial label on target domain) is not a strong contribution and it has been only validated on toy(ish) MNIST problems.
Based on these points, we can not recommend acceptance.
justification_for_why_not_higher_score: This paper is incremental compared to Korotin 2022c. indeed, it is more general but the related properties and algorithm can be easily derived from that paper.
justification_for_why_not_lower_score: NA
Decision: Reject