COT Flow: Learning Optimal-Transport Image Sampling and Editing by Contrastive Pairs

Authors: Xinrui Zu, Qian Tao
Primary area: generative models
Keywords: generative models, consistency models, diffusion models, optimal transport

Abstract:
Diffusion models have demonstrated strong performance in sampling and editing multi-modal data with high generation quality, yet they suffer from the iterative generation process which is computationally expensive and slow. In addition, most methods are constrained to generate data from Gaussian noise, which limits their sampling and editing flexibility. To overcome both disadvantages, we present Contrastive Optimal Transport Flow (COT Flow), a new method that achieves fast and high-quality generation with improved zero-shot editing flexibility compared to previous diffusion models. Benefiting from optimal transport (OT), our method has no limitation on the prior distribution, enabling unpaired image-to-image (I2I) translation and doubling the editable space (at both the start and end of the trajectory) compared to other zero-shot editing methods. In terms of quality, COT Flow can generate competitive results in merely one step compared to previous state-of-the-art unpaired image-to-image (I2I) translation methods. To highlight the advantages of COT Flow through the introduction of OT, we introduce the COT Editor to perform user-guided editing with excellent flexibility and quality.

Decision: reject
Reviewer scores: 5, 3, 5

Reviews and discussion:
<thread>
creation date:
1730107275167

thread_type:
official_review

<thread_object>
creator: reviewer_v1md

<content>
summary:
This work presents Contrastive Optimal Transport Flow (COT Flow), a new method that achieves fast and high-quality generation with improved zero-shot editing flexibility compared to previous diffusion models.

strengths:
This work presents Contrastive Optimal Transport Flow (COT Flow), a new method that achieves fast and high-quality generation with improved zero-shot editing flexibility compared to previous diffusion models.

weaknesses:
1. Experiments are not enough only on handbag→shoes (64×64), CelebA male→female (64×64), and outdoor→church (128×128). These tasks are unuseful.
2. The images are too small and unclear.
3. The comparison methods are too old. It should compare with at least some of the latest text-based image editing methods in 2024.

questions:
1. How to use this flow to open-set text-based image editing? What is the source distribution and target distribution?
2. What is the role of \phi_\omiga in Eq.10 and 11? Eq.10 and 11 confuse me.
3. In Algorithm 1 COT Training, if the encoder learns to output all zero, the loss function will be zero, how to handle this problem?

I will be happy to raise the rating if the response is good.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730441737115

thread_type:
official_review

<thread_object>
creator: reviewer_ppkv

<content>
summary:
This paper proposes the COT Flow model by integrating neural optimal transport and consistency models. Based on the similarities between contrastive learning and consistency models, the authors introduce a new method for defining positive pairs. Furthermore, they demonstrate that the COT Flow can be applied to zero-shot image editing.

strengths:
1. The proposed COT Flow model effectively enables unpaired image-to-image translation.
2. The COT Flow model shows potential for various zero-shot image editing scenarios.

weaknesses:
1. Comparison methods are limited.
2. Quantitative evaluations are limited, relying solely on FID scores.
3. Qualitative results lack visual impact.

questions:
1. Does COT Flow truly address the generative learning trilemma? The comparison only includes FID scores, but if the model claims to tackle the trilemma, it should demonstrate comparisons in terms of sampling speed, quality, and diversity against existing methods.
2. What is the training process for the neural optimal transport model?
3. Comparison methods are limited. Numerous recent studies on diffusion-based unpaired image-to-image translation tasks should be included, such as [1], [2], and [3]. Additional comparisons with more recent or relevant baselines could strengthen the validity and impact of the findings.

[1] Korotin, A., Selikhanovych, D., & Burnaev, E. Neural Optimal Transport. In The Eleventh International Conference on Learning Representations.
[2] Su, X., Song, J., Meng, C., & Ermon, S. Dual Diffusion Implicit Bridges for Image-to-Image Translation. In The Eleventh International Conference on Learning Representations.
[3] Zhao, M., Bao, F., Li, C., & Zhu, J. (2022). Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. Advances in Neural Information Processing Systems, 35, 3609-3623.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730738289077

thread_type:
official_review

<thread_object>
creator: reviewer_eumd

<content>
summary:
This paper present Contrastive Optimal Transport Flow (COT Flow), a method that achieves fast and high-quality generation with improved zero-shot editing flexibility compared to previous diffusion models.

strengths:
The paper is well-written

weaknesses:
1.The performance improvement of the paper is not significant.

2.The comparison method is outdated; SDedit is a work from two years ago.

This paper has neither impressive results nor significant improvements. I did not find any highlights in this paper, leaning towards a rejection.

questions:
see above
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces the Contrastive Optimal Transport Flow (COT Flow), aimed at improving the speed and quality of image generation, with a focus on zero-shot editing flexibility. The model integrates neural optimal transport with consistency models for defining positive pairs in contrastive learning.

***Strength:***
- The paper is clearly written and presents a novel integration of neural optimal transport with consistency models.
- Demonstrates potential for application in zero-shot image editing scenarios.

***Weaknesses:***
- The performance improvement over existing methods is marginal.
- Uses outdated comparison methods and lacks significant comparative analysis with recent state-of-the-art models.
- Limited scope and depth in experiments which only on small-scale tasks with low-resolution images.
- Both qualitative and quantitative evaluations are weak, failing to provide compelling evidence of the model's effectiveness.

The authors did not submit a rebuttal to address the concerns raised during the review process, missing the opportunity to clarify the highlighted issues. Given all the concerns remaining, this paper cannot be accpeted at this time.