Title: Balanced Conic Rectified Flow
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generative models

Flow model

Optimal transport

Curvature

Ordinary Differential Equation (ODE)
Abstract: Rectified flow is a generative model that learns smooth transport mappings between two distributions through an ordinary differential equation (ODE). The model learns a straight ODE by reflow steps which iteratively update the supervisory flow. It allows for a relatively simple and efficient generation of high-quality images. However, rectified flow still faces several challenges. 1) The reflow process is slow because it requires a large number of generated pairs to model the target distribution. 2) It is well known that the use of suboptimal fake samples in reflow can lead to performance degradation of the learned flow model. This issue is further exacerbated by error accumulation across reflow steps and model collapse in denoising autoencoder models caused by self-consuming training.

In this work, we go one step further and empirically demonstrate that the reflow process causes the learned model to drift away from the target distribution, which in turn leads to a growing discrepancy in reconstruction error between fake and real images. We reveal the drift problem and design a new reflow step, namely the conic reflow. It supervises the model by the inversions of real data points through the previously learned model and its interpolation with random initial points. Our conic reflow leads to multiple advantages. 1) It keeps the ODE paths toward real samples, evaluated by reconstruction. 2) We use only a small number of generated samples instead of large generated samples, 600K and 4M, respectively. 3) The learned model generates images with higher quality evaluated by FID, IS, and Recall. 4) The learned flow is more straight than others, evaluated by curvature. We achieve much lower FID in both one-step and full-step generation in CIFAR-10. The conic reflow generalizes to various datasets such as LSUN Bedroom and ImageNet.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose conic rectified flow (RF), a new alternative to RF to mitigate the distribution drift in the successive reflow steps. The main idea of the conic RF is to design a map that transports a neighbourhood of points (rather than a single Gaussian point) to a corresponding image point $X_1$.\nThe experimenst show that conic RF is able to reduce the reconstruction error gap between real and fake images, and also achieves comeptetive results in real examples.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\nThe idea seems novel and quite effective in practice. To generate reliable results in practice, it is essential to address and resolve the issue of distribution drift in the RF reflow steps. This paper attempts to provide a solution in that direction. The experiments presented in the paper are comprehensive and showcase the usefulness of conic RF.\n\n## Weakness \nAlthough the idea is novel, the exact benefit of the current approach over regular RF is not clear. For example, how exactly conic RF prevents the distribution drift is not clear to me immediately. All in all, I think a stronger case needs to be made in favor of conic RF to argue the significance of conic RF over other methods (e.g., RF with real data augmentation introduced in [1] )\n Also, there are some presentation issues that I have pointed out later.\n\n\n[1] Zhu, H., Wang, F., Ding, T., Qu, Q., & Zhu, Z. (2024). Analyzing and Improving Model Collapse in Rectified Flow Models. arXiv preprint arXiv:2412.08175.\n\nQUESTIONS:\n## Major comments\n**Definition of $v$ is not clear:**\n- Line 72: ''coupling the generated pairs $(Z_0, v(Z_0))$\nensures that the interpolation trajectory preserves the marginal distributions of the original and target\ndomains'': Assuming $v$ is the velocity field,  I think $(Z_0, v(Z_0))$ does not preserve the marginals of the source and target distribution. In fact, $v(Z_0)$ is just the velocity field (the time argument is missing). I think it should be $(Z_0, T(Z_0))$, where $T(Z_0):= Z_0 + \\int_{0}^1 v(Z_t, t)\\; dt$. Do the authors mean to write $T$ instead of $v$?\n- Line 87: Footnote 1 suggest that the $t$-th sampling step is denoted by $v(X_0)$ which shoudl be $X_t + v(X_t, t)dt$ (Not $X_0 + v(X_t, t)dt$) as per my understadning. Is it denoting something else?  Also, $v(Z_0^k)$ seems to denote the last sampling step instead of any $t$th sampling step. The notations are hard to understand. \n\n- Shouldn't there be an integral over $\\epsilon, \\zeta$ in Eq(9)?\n\n\n**What exactly is better in conic RF than regular RF?:**\n\n- Section 3.1 argues that RF suffers from distribution drift, i.e., $T(T^{-1}(X_1)) \\ne X_1$. How is this getting resolved in the Conic RF? As far as I understand, conic RF is trying to map rather an expanded neighbourhood around $T^{-1}(X_1)$ to a single point $X_1$. But, how does it stop the distribution shift? \n\n- Prior works have shown that real data augmentation reduces model collapse. Conic RF also uses real data during training, along with fake data. It might be possible that real data augmentation is the main factor that is preventing the distribution drift, rather than the explicit design of the flow mechanism. For example, [2] shows that real data augmentation mitigates model collapse. In light of this fact, it seems that using the methodology in [2] is more practical than opting for conic RF, as conic RF is comparatively harder to train. What is the extra advantage of Conic RF? I think a discussion on this is important. Maybe adding a comparison between the method in [2] and conic RF will help.\n\n\n## Minor comments\n\n- Line 67 writes k-rectified flow, whereas Line 74 writes $k$-rectified flow. It is advisable to choose one consistent notation.\n\n- The set $\\mathbb{N}$ in Line 160 is no defined. Also, where does the index $i$ in Eq (12) occur in Eq (11)? Are you referring to the empirical version of the loss in Eq. (11)?\n\n[1] Liu, X., Gong, C., & Liu, Q. (2022). Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003.\n\n[2] Zhu, H., Wang, F., Ding, T., Qu, Q., & Zhu, Z. (2024). Analyzing and Improving Model Collapse in Rectified Flow Models. arXiv preprint arXiv:2412.08175.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses ReFlow, which is a procedure for straightening flow trajectories by training flow models upon noise-image pairs generated by itself. While ReFlow does produce faster flow models, the generated marginal drifts away from true marginals as a result of training with fake pairs (pairs generated by solving the flow starting from noise to data). While training with real pairs (pairs generated by solving the flow starting from data to noise) can alleviate this drifting problem, one can only generate a finite number of real pairs.\n\nTo mitigate this problem, the authors propose to augment real with with **conic pairs**. Given a real pair, a conic pair is produced by spherically interpolating generated noise with random noise. ReFlow with conic pairs is shown to out-perform vanilla ReFlow on CIFAR10, ImageNet 64x64, and LSUN Bedroom datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths and weaknesses are denoted [S#] and [W#], respectively.\n\n**Quality**\n- [S1] The paper provides thorough ablations along with experiments on standard benchmark datasets such as CIFAR10, ImageNet, and LSUN Bedroom.\n\n**Significance**\n- [S1] Conic ReFlow may be applied to larger flow models such as InstaFlow and SD3.\n- [W1] The performance of conic ReFlow is rather weak when compared to recent ReFlow models such as [2] or [3]. For instance, on CIFAR10, while conic ReFlow achieves 3.24 FID with NFE=104, [2] achieves 1.98 FID with NFE=9, and [3] achieves 3.07 FID with NFE=1. On ImageNet 64x64, while conic ReFlow achieves 28.2 FID with full simulation (NFE unreported), [2] achieves 1.74 FID with NFE=9, and [3] achieves 4.31 FID with NFE=1.\n\n**Clarity**\n- [S1] The experimental protocol is described clearly.\n\nI had difficulty following the logic at some parts of the paper. See below for details.\n\n- [W1] At lines 104-109, the authors assert that continuity near real samples imply closeness of generative and real marginals. Why should this be the case? The former property is a local one, and the latter property is a global one, and in general, one does not imply the other.\n- [W2] At lines 146-148, the authors claim \"this approach (conic ReFlow) not only mitigates distributional drift but is also aligned with adversarial robustness strategies studied in inverse problems and reconstruction-based neural networks\". Given the ambiguity described in [W1], I have difficulty seeing how conic ReFlow mitigates distributional drift.\n- [W3] In Section 4.4.1, the authors propose refreshing real pairs every fixed number of model updates. However, it is unclear why one should do so. Wouldn't it be better just to generate a large number of real pairs prior to training?\n\n**Originality**\n- [S1] This paper proposes a new type of augmentation for real pairs for ReFlow training.\n- [W1] A similar augmentation has been explored in [1] (albeit with different motivations). Specifically, [1] proposes annealing ReFlow, where, given a noise-image pair, one interpolates noise with random gaussian noise.\n- [W2] The idea of balancing training with real and fake pairs has also been suggested in [2].\n\n[1] SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow, ECCV, 2024.\n\n[2] Simple ReFlow: Improved Techniques for Fast Flow Models, ICLR, 2025.\n\n[3] Improving the Training of Rectified Flows, NeurIPS, 2024.\n\nQUESTIONS:\nI am willing to raise the score to *borderline reject* if the authors address concerns in\n\n- **Clarity** with further explanations or proofs,\n- **Originality** by adding proper baselines,\n\nand further to *borderline accept* if the authors address concerns in\n\n- **Significance** by demonstrating that conic ReFlow can achieve results (e.g., FID scores) comparable to those of recent ReFlow models such as RF++ and Simple ReFlow.\n\nLIMITATIONS:\nThe authors discuss some limitations in the conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article reveals that the reflow process causes the learned model to drift away from the target distribution and designs a new reflow step, namely the conic reflow, to address this drift problem. It supervises the model by the inversions of real data points through the previously learned model and its interpolation with random initial points. Conic reflow achieves much lower FID in both one-step and full-step generation in CIFAR-10. The conic reflow generalizes to various datasets such as LSUN Bedroom and ImageNet.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\nThe article is well written and organized. The article proposes a new type of reflow, named conic reflow. It keeps the ODE paths aligned with real samples, as evaluated by reconstruction. It uses only a small number of generated samples, 600K and 4M, respectively, instead of the large ones. The learned flow is straighter than others, as evaluated by curvature. Experiments demonstrate that learned model generates images with higher quality, evaluated by FID, IS, and Recall.\n\n\n\n# Weaknesses\n\n1. The proposed method lacks a thorough theoretical analysis of the conic flow and its potential to alleviate drift issues. Additionally, the paper does not include a comprehensive analysis for selecting slerp or conic hyperparameters theoretically or empirically. \n\n2. The paper compares performance only with the original rectified flow and does not include a comparison with recent flow matching models or diffusion models.\n\n3. The empirical improvements for the conic reflow in CIFAR-10 are too small, and it does not adequately verify the effectiveness of the conic reflow. The author should include more empirical results (such as higher-resolution ImageNet, FFHQ, AFHQ, or other classes of LSUN).\n\nBased on the above content, I recommend that the author submit additional experimental results for review. If you can provide additional theoretical analysis for the conic reflow, it will also help change the perceptions of the paper.\n\nQUESTIONS:\n1、Please refer to the question in the weakness section.\n\n2. The ablation study of selecting slerp or conic hyperparameters is not sufficient. The metric differences between different hyperparameter combinations are too small.\n\n3. I wonder if the conic reflow costs more compute resources than the original Rectified Flow?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper first demonstrates that the original reflow process in Rectified Flow relies on large amounts of generated data and can cause drift from the target distribution. To address this, the authors propose Balanced Conic Rectified Flow, which incorporates real image inversions and spherical linear interpolation. This approach achieves better performance while requiring significantly fewer generated samples.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n- The proposed method outperforms reflow and rf++, with experiments on diverse datasets including CIFAR-10, ImageNet, and LSUN.\n- The drift analysis is insightful and serves as a good motivation. \n\nWeaknesses:\n- The paper analyzes drift using 1-step Euler sampling; however, Rectified Flow models are not designed for 1-step generation, and fake samples for the reflow process are generated via multi-step integration. This raises concerns that the measured drift may reflect Euler integration error rather than actual model drift, potentially biasing the conclusions. Clarification or validation with a multi-step analysis would strengthen the argument. \n- The proposed method introduces several hyperparameters, including the maximum Slerp interpolation weight $\\zeta^{max}$, the repairing step, and the mixing ratio between real and fake pairs in balanced conic reflow. However, the paper lacks sufficient analysis of how sensitive the model's performance is to these hyperparameters. It is unclear whether the reported improvements are robust or heavily dependent on careful tuning. \n- While the paper empirically demonstrates that using Slerp-based perturbations around real image inversions helps mitigate drift, it does not offer a theoretical explanation for why this technique effectively addresses the issue.\n\nQUESTIONS:\nSee Weakness. Also, \n- The arrangement and legends in Figures 3a–b and 5b could be improved for better readability. \n- The first footnote on page 3 is somewhat confusing. It refers to \"the t-th sampling step\", and adds the velocity at time $t$ directly to $X_0$ or $X_1$, which does not make sense. It seems that the process being described should involve integration from $t = 0$ to $t = 1$, or vice versa, rather than referring to a single \"t-th sampling step\".\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}