Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose InfoOT which maximizes the mutual information between domains and minimize distances between input distributions. The proposed InfoOT address several drawbacks of OT, e.g., coherence structure (clustering, outliers) and easy to integrate new data points. Empirically, the authors evaluate the proposed method on domain adaption, cross-domain retrieval and single-cell alignment.
strength_and_weaknesses: Strength
+ The proposed InfoOT address several drawbacks of OT 
+ The proposed method works well in applications.

Weaknesses
+ The advantage of the proposed method (using kernel density estimation for continuous setting) is not clear enough over discrete ones (e.g., entropic regularization in Sinkhorn), especially in the case input distributions are discrete (empirical distributions). It will be a plus if the authors elaborate this points with more details.
+ It is unclear how many samples are required for the kernel density estimation used in InfoOT
clarity,_quality,_novelty_and_reproducibility: The proposed ideas are interesting. The proposed method, InfoOT can address several drawbacks from OT, especially about coherence structure (cluster, outlier) and the ability to integrate new data points.

I have some following concerns:
+ In case the input distributions are discrete, e.g., empirical distributions. It is not clear the advantages of using kernel density estimation (as in the proposed method) comparing to the entropic regularization (in Sinkhorn) for measuring global structure with mutual information (as in Section 4.1). Could the authors elaborate it with more details?

+ How many samples are required for the kernel density estimation (KDE)? and how to choose the bandwidth for the Gaussians used in KDE for the proposed InfoOT? especially for high-dimensional setting?

+ Could the author discuss the relation between the proposed InfoOT with Liu'2021 which is also based on mutual information and OT from given unpaired data?

+ For the robustness against noisy data, it is better if the authors compare the proposed method with the unbalanced OT approach (which also use to address this problem for OT). Could the authors discuss about it (and better to have some empirical comparison)?

+ In experiments, it is well-known that the entropic regularization affects performances of entropic OT, why the authors set it to 1 in experiments?
--- How the \lambda in Fused InfoOT affects its performances in applications? Why the authors set it to 100? Should one need to use \lambda to control the effect of the regularization?
summary_of_the_review: The proposed method is interesting. The proposed methods address several drawbacks of OT.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed to improve the quality of optimal transport via simultaneously encouraging the mutual information (MI). This is justified by establishing the equivalence between the MI and the standard entropy regularization used in the Sinkhorn algorithm. Empirical results showed that when implemented with kernel density estimation, the proposed InfoOT better captures the cluster structure in data and works better on serval domain transfer tasks. An alternative view on the smoothness regularization is also given.  The author(s) further presented a conditional projection scheme for the OT mapping. 
strength_and_weaknesses: 
### Strength
* Calling out the equivalence between entropy regularization and the proposed MI optimization makes an interesting point. 
* The smoothness argument to preserve cluster structure makes good sense. 
* Reported empirical gains on domain adaption and cross-domain retrieval are very encouraging

### Weakness
* As a criticism applies all kernel methods, the main concern is scalability of the method. First, the computation scales quadratically, and to ensure good performance, heavy parameter tuning is typically needed. Integration and end-to-end optimization with need neural nets (which is known to perform strongly even without the kernel trick), is often impractical with kernel-based components. 
* There are existing work on the neural estimation of the transport plan and non-parametric estimation of MI (e.g., InfoNCE, NWJ, etc.). While integrating these two directions as a generalized version of the proposed solution seems to be out-of-scope, I would love to see some technical discussions at least. 
* While I appreciate the the cluster-preserving idea, this is merely a flaky heuristic assumption and may not hold universally. Also some ablation study using alternative, more direct penalties to enforce smooth transport should be considered (e.g., the distance between nearby source points in the transported target space). 
* For the domain adaption experiment, while I understand the author(s) followed the experiment setup from prior works, but 1-NN classification is usually considered unreliable. Numbers with alternative classification schemes should be reported (e.g., 5-NN, linear, simple neural net, etc.)
* There is no discussion on the limitations of the proposed method

clarity,_quality,_novelty_and_reproducibility: ### Clarity. Very good.
I really appreciate the clarity of the presentation. 

### Quality. Okay
This paper is well-written. Optimal transport is a versatile tool and there is growing interests in applying OT-based solutions to address various empirical problems. This work has exploited a simple smoothness heuristic to address the cluster preserving issue common to many OT applications. To make the points more intuitive the author(s) have presented visualization with both synthetic and real datasets. 

Relevant literature is adequately cited. It will be nice to further complete the picture with refs on the more generic integral probability metric (IPM) and generalizations of OT such as Sinkhorn divergence. Also potentially compared with the more stable OT algorithms such as Inexact Proximal point method for Optimal Transport (IPOT), my hunch is that vanilla Sinkhorn is known to be unstable, and that might be the cause of issues discussed in the paper. Using more stable OT algorithms may address these pain points without any smoothness regularization. 

### Novelty. Fair
KDE is a well-establish technique, and smoothness is a common regularity widely applied in machine learning. 

### Reproducibility. Good
Details of the solution and experiment setups are very well documented, and there should be no major difficulty involved implementing the model. 

summary_of_the_review: This paper is well-written. My major reservation is due to the lack of technical novelty and practical significance. KDE is a well-establish technique and it does not scale well, also for the problems considered in the experiments there should be better alternative solutions without applying OT (especially for the domain adaptation). I am open to reconsider my decision should more convincing argument or empirical results surface during the rebuttal phase.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a dynamic cost function for OT (see below on whether this is a valid OT objective or not) that attempts to maximize the mutual information of the projections using kernel density estimates---which depend on the coupling distribution.
This can be seen as a generalization of entropic regularization.
The InfoOT formulation can be merged with the standard OT problem and can then be solved via a sequence of entropic OT problems.
Empirically, this is shown to preserve cluster structure in 2D examples and qualitative t-SNE projections.
Furthermore, the paper proposes a mapping method based on the kernel density estimates called conditional projection that is more robust to projecting outliers.

strength_and_weaknesses: *Strengths*
- The proposed training method can maintain cluster structure when learning a mapping.

- The proposed mapping method can be more robust to outliers than barycentric proojection.

- The algorithm simplifies to a sequence of entropic OT problems.

- The results show improved performance compared to standard OT baselines across several tasks.

*Weaknesses*
- Kernel densities are known to breakdown in higher dimensions and suffer from the curse of dimensionality. This presents a key limitation and problem with the proposed approach. Why would this not be a problem here?  While there are experiments on MNIST for domain adaptation, it is unclear what is actually going on. 

- The baselines for all external tasks seem to be OT-based. What about non-OT methods for these tasks?  For example, what about GAN-based methods for domain adaptation (e.g., [Ganin et al. 2016]) or Wasserstein dual formulations for learning OT maps via convex functions (e.g., [Makkuva et al., 2020])? There are also flow-based methods for alignment that could be used for domain adaptation [Grover et al. 2020, Usman et al., 2020]. I am unfamiliar with the sota methods for retrieval but it seems there would also be methods here. While I understand that you are comparing to other OT-based methods, the key motivation for this estimator seems to be for empirical reasons.  It is unclear that there is theoretic or "intrinsic" motivation for the proposed OT formulation. Thus, to demonstrate it's usefulness more broadly, it would be helpful to compare to sota methods for these tasks and/or do more intrinsic evaluation.

- Definition 1 seems odd or incorrect. It seems that Definition 1 is using two approximations that should be distinguished and explained. First, you approximate the joint distribution over x,y using a kernel density estimate.  If you were to use a plug-in estimator, this would still require an expectation using the kernel density for the expectation.  However, then you replace this expectation over the kernel density, with a psuedo-sample approximation that treats each possible pair of inputs (x,y) as a weighted empirical sample from this kernel density. Thus, the expectation turns into a weighted average with $\Gamma$ defininig the weights of each pairing.  I believe this is how you simplify to only using $\Gamma$ as the weights for the outer summation in Definition 1. Again, these two approximations should be discussed and distinguished as it was not clear from the original exposition.

- Using 1-NN in the source domain as the target predictor for domain adaptation and using KNN in the retrieval experiment are unlikely to do well in high dimensions and may be biased towards KDE-based non-parametric approaches. A standard CNN or fully-connected network could be used in evaluating domain adaptation. In particular, a KNN classifier may be unfairly biased towards KDE-like approaches as they are both non-parametric methods based on geometric distances.

- I'm not sure it is reasonable to have $\Gamma$ in the "cost" function of OT at least theoretically. It's unclear if it's actually OT anymore if $\Gamma$ is in the cost function itself.  Could  you explain why this is still OT if the cost function is now a function of the map itself? 

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., ... & Lempitsky, V. (2016). Domain-adversarial training of neural networks. The journal of machine learning research, 17(1), 2096-2030.

Makkuva, A., Taghvaei, A., Oh, S., & Lee, J. (2020, November). Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning (pp. 6672-6681). PMLR.

Grover, A., Chute, C., Shu, R., Cao, Z., & Ermon, S. (2020, April). Alignflow: Cycle consistent learning from multiple domains via normalizing flows. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 04, pp. 4028-4035).

Usman, B., Sud, A., Dufour, N., & Saenko, K. (2020). Log-likelihood ratio minimizing flows: Towards robust and quantifiable neural distribution alignment. Advances in Neural Information Processing Systems, 33, 21118-21129.

clarity,_quality,_novelty_and_reproducibility: - Can this be interpreted as an adaptive/dynamic cost function for OT? This is related to the weakness above. Overall, it seems odd and non-standard but may be an interesting idea that could be generalized. 

- Could the KDE weighting function be used as a post-hoc smoothing method for standard OT?  It seems that it could be applied after doing empirical/entropic OT to provide smoother mappings.  If so, could you add it to the baseline methods?  This would help distinguish the mapping method from the OT training method.

- The discussion in Section 4.1 on discrete vs continuous may be premature.  It might be useful to discuss this later in the paper.

- Does equation 1 have a typo, it seems that the second equation should be $d_{\mathcal{Y}}$.

- Table captions should be on top of tables.

summary_of_the_review: Overall, I found the paper to address two interesting empirical problems related to using OT in ML: Preserving cluster structure and outliers. The technical descriptions were a bit vague but the overall idea of adding a mutual information regularizer seems interesting. However, I do have some concerns about the use of KDEs because they can break down in high dimensions, the experimental setup, and some technical points. I hope that at least some of these can be addressed by the author response.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: In this paper, the authors propose a new optimal transport based on information maximization and name it infoOT. More specifically, the authors combine kernel density estimation (KDE) and optimal transport. The infoOT framework is interesting. However, the original idea of information maximization OT has existed since 2000s. For example, kernelized sorting (NIPS 2008) is highly related to the infoOT work. Moreover, as pointed by a reviewer, the comparison of Liu 21's work is also needed. Thus, I encourage the authors to update the paper based on the reviewer's comments and resubmit it to a future ML venue.





justification_for_why_not_higher_score: The paper is interesting. However, the relationship to the existing methods is still not clear. Moreover, the positive reviewers are not excited to publish at this point.
Decision: Reject