Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching
Authors: Junho Lee
Kwanseok Kim
Joonseok Lee
Venue: Accepted by TMLR ()

Abstract: Flow matching has emerged as a powerful generative modeling approach with flexible choices of source distribution. While Gaussian distributions are commonly used, the potential for better alternatives in high-dimensional data generation remains largely unexplored. In this paper, we propose a novel 2D simulation that captures high-dimensional geometric properties in an interpretable 2D setting, enabling us to analyze the learning dynamics of flow matching during training. Based on this analysis, we derive several key insights about flow matching behavior: (1) density approximation can paradoxically degrade performance due to mode discrepancy, (2) directional alignment suffers from path entanglement when overly concentrated, (3) Gaussian's omnidirectional coverage ensures robust learning, and (4) norm misalignment incurs substantial learning costs. Building on these insights, we propose a practical framework that combines norm-aligned training with directionally-pruned sampling. This approach maintains the robust omnidirectional supervision essential for stable flow learning, while eliminating initializations in data-sparse regions during inference. Importantly, our pruning strategy can be applied to any flow matching model trained with a Gaussian source, providing immediate performance gains without the need for retraining. Empirical evaluations demonstrate consistent improvements in both generation quality and sampling efficiency. Our findings provide practical insights and guidelines for source distribution design and introduce a readily applicable technique for improving existing flow matching models. Our code is available at https://github.com/kwanseokk/SourceFM.

Published in Transactions on Machine Learning Research (12/2025)
Is There Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching
Junho Lee" Graduate School of Data Science Scoul National Universily
joon2003
Kwanscok Kim
1jpd10090@eraau oc.kr
Graduate School of Data Science Seoul National University
Joonseok Leef
joonseok@snu..ok oc.kr
Graduate School of Data Science Seoud National University
Reviewed on OpenReview: https: // opw.reveeeereeeee..om net/ forum? id= sevOGtVife
Abstract
Flow matching has emerged as a powerful generative modeling approach with flexible choices of source distribution. While Gaussian distributions are commonly used, the potential for better alternatives in high-dimensional data generation remains largely uncxplored. In this paper, propose novel 2D simulation that captures high-dinensiioall geometric properties in an interpretable 2D setting, enabling us to analyze the learning dynamics of flow matching during training. Based on this amalysis, we derive several key insights about flow matching bchavior: (1) density approximation can paradoxically degrade performance due to mode discrepancy, (2) directional aligument suffers from path entanglement when overly concentrated, (3) Gaussian's 's omnidirectional coverage ensures robust learning, and (4) norm misalignment incurs substantial learning costs. Building on these insights, propose sampling.
practical framework that combines norm-aligned training with directionally-prun This approach maintains the robust omnidirectional supervision essential for stable flow learning, while eliminating initializations in data-sparse regions during inference. Importantly, our pruning strategy can be applied to any flow matching model trained with Gaussian source, providing immediate performance gains without the need for retraining. Empirical cvaluations demonstrate consistent improvements in both generation quality and sampling efficiency. Our findings provide practical insights and guidelines for source distribution design and introduce readily applicable technique for improving existing flow matching models. Our code is available at https://githhn http:://ggthub..oo///aoe wwwoo//wwwaoo////SA.. /SourccF..
1 Introduction
Flow Matching (FM) (Lipman
2023 Liu et al.
2023b Albergo al. 2023 a recently introduced
approach for generative modeling that bridges ideas from Continuous Normalizing Flows (CNFs) CCen et al.
2018 Grathwohl et al. 2018) and diffusion models (Sohl-Dickstein et al. 2015 Song & Ermon 2019 Ho
et al. 2020 Nichol & Dhariwal 2021 Song et. al.] (021b). The core idea. involves training a vector field so that trajectories, following an ordinary differential equation (ODE), transport samples from a simple base distribution to a complex target distribution, offering advantages in stable training, fast inference, and explicit likelihood estimation. Unlike diffusion models, which typically involve a fixed stochastic process that
"Equal contribution.
i "Correpponding author.

---

Published in Transactions on Machine Learning Research (12/2025)
Source
Source
Source we
Generated
Generated
Genérated Data
Data
Data
(b)
(c)
Figure 1: 2D simulations of flow matching. The first two are examples of naive low-dimensional flow matching illustrations of (a) Gaussian to Gaussian and (b) 8 Gaussians to moons. Our proposed 2D simulation for flow matching is demonstrated in (c). Dots represent source (black), target data (blue), and generated samples (red). Blue dashed lines show successful ODE trajectories from source generated samples, while red dashed lines indicate failed trajectories.
gradually adds Gaussian noise to data and requires learning a reverse denoising trajectory, flow matching offers complete freedom in defining the interpolation path between distributions. This flexibility not only enables the design of more efficient and straighter trajectories but also removes any constraint on the choice of source distribution.
This flexibility has motivated extensive research on enhancing FM performance through improved source source-targel pairing strategies (Tong et al.
2024
Pooladian et al.
2023
Davtyan et al. 2025) and developing straighter
probability paths (Xing et al.
2023
Lee et al. 2023 Liu et al. 2023b to reduce trajectory curvature and
minimize function evaluat ions. Recent advances have leveraged prior knowledge of target distributions to
construct better-aligned source distributions (Stark et al.] 2023 Jing et al. 2024 Kollovich et al. 2025).
demonstrating substantial improvements through partial target information incorporation. For multimodal generation, recent works have exploited this source flexibility by using text embeddings directly source
distributions (Liu et al. 2024 He al. 2025
However, most existing target-aware source designs are still limited to relatively simple or low-dimensional datasets, such as time series [Kollovieh et al. 2025) or protein-ligand interactions (Stark et al. 2023). To the best of our knowledge, there has been little to no prior work applying them to complex, high-dimensional data like natural emages-likell) due to the challenging nature of such applications. Moreover, naively replacing the standard Gaussian source with distributions from other modalities (e.g., text embeddings) often fails achieve stable target mappings, requiring extensive engineering as shown by prior work (Liu al. 2024 He et al. 2025). This raises fundamental question: What makes deviating the source from the Gaussian distribution difficult, and what properties are required for source to be advantageous in flow matching? To answer this, we begin with an intuitive hypothesis: source distributions that more closely approximate the density or geometric characteristies of the target distribution would lead to improved performance and faster convergence.
Since learning behavior in high-dimensional datasets is difficult to interpret and visualize, we begin with low-dimensional experiments that enable clear visual analysis of the underlying learning dynamics. Several prior works have used naive low-dimensional experiments to demonstrate their effectiveness, c.g., 2D Gaussian to 2D Gaussian in Fig. a 8 Gaussians to moons in Fig. b).. However, discover that merely using low-dinensionality does not reflect the intricate but essential learning dynamics that emerge only in high-dinensional cases, and thus lessons learned from such a naive low-dimensional loy experiment do not fully transfer to the actual applications of interest, which typically involve high-dimensional data. To address this, we rodesign the low-dimensional experiments in Section 3 to better reflect the geometric properties of dimh-dimensional data by decomposing it into direction and norm components, as illustrated in Fig. (cc. This 2D simulation allows to more realistically analyze the learning dynamics of flow matching, preserving
2

---

Published in Transactions on Machine Learning Research (12/2025)
the essential characteristics of high-dimensional applications while enabling clear visualization of vector field behaviors in 2D.
Using this novel experimental framework, we examine in Section how different source distribution strategies interact with various pairing methods (independent us. in-batch optimal transport). This allows us to test. our intuitive hypothesis that source distributions more closely approximating the density or geometric characteristies of the target distribution should improve performance. Specifically, we propose three strategies to redesign the source distribution: progressively approximating the density of the target distribution (density approzimation), designing sources based directional information of the target distribution (directional alignment), and aligning the expected norms of the source and target distributions (norm aligromet)).
From our sinulation results and analysis in Sections 1 and 5 we derive several critical insights that reshape our understanding of source distribution design in flow matching: (1) Source distributions closely approximating the target density can paradoxically degrade performance, with stronger approximations lcading worse outcomes phenomenon we altribute to mode discrepancy where approximale sources omit low-density data modes. (2) Directional aligmment strategies, while theoretically appealing, often suffer from path entanglement due to insufficient source support and imperfect pairing, both of which hinder optimal performance. (3) The standard Gaussian source with independent pairing demonstrates unexpected robustness through omnidirectional coverage, while in-batch optimal transport pairing, despite its efficiency, sacrifices this crucial omnidirectional learning. (4) Regions with sparse or missing data consistently lead to generation failures due to insufficient supervision of the vector field during training. (5) A siguificant discrepancy between the norm distributions of the source and target incurs substantial learning cost to resolve. These patterns hold consistently across both our controlled 2D simulations and real-world high-dimensional image datasets, demonstrating that our framework captures fundamental dynamics of flow matching.
Our key findings suggest that an effective source distribution design should leverage specific advantages while mitigating known weaknesses. Accordingly, we propose a new hybrid framework that combines training on a Gaussian source with "Norm Aligument" and employing "Pruned Sampling" during inference. This approach preserves the benefits robust, omnidirectional learning from the Gaussian source (Finding 3) while resolving the learning inefficiency caused by norm mismatch (Finding 5). Furthermore, Pruned Sampling directly tackles the issue of passing through regions where the vector field is inaccuratcly learned (Finding 4) by excluding sampling from data-sparse regions, thus improving the quality of the final output. Notably, Pruned Sampling can be applied post-hoc to improve the performance of existing, pre- trained models without any need for retraining. This provides a practical pathway to immediately upgrade a wide range of flow matching models.
To summarize, we first propose a novel set of 2D simulation experiments designed to reveal and interpret the complex learning dynamics of flow matching in high-dimensional settings. Through these experiments, we derive critical findings that lead to set of practical guidelines for source distribution design. Finally, we propose readily applicable pruned sampling technique that can significantly improve existing flow matching models without the need for retraining. of the flow matching field but also suggest new directions for developing more efficient and robust generalive models.
We believe these contributions not only advance the understanding
2 Background and Related Work
2.1 Flow Matching
Continuous Normalizing Flows (CNFs) CChe et al.
2018 Grathwohl et al. 2018 define an ODE dz = ur (z)dt
Rdd that transports an initial distribution go to target distribution over E [0, 1]. This ODE induces a path of densities (x) satisfying the continuity equation (x) + V (p:(z) (z)) = O. Flow Matching (FM) (Albergo et al. 2023 Lipman et al. 2023 Liu et al., 2023b) is a simulation-free approach for training CNFs that avoids trajectory simulation by regressing the model's vector field a known probability flow. Instead of maximizing log-likelihood, FM assumes prescribed probability path pz( that smoothly interpolates from po 4o Pı Qı, along with its
with a time-varying velocity field u 10. x R
3

---

Published in Transactions on Machine Learning Research (12/2025)
corresponding velocity field u, (x). The neural ODE (9 t) is trained to match ue(x) via
(r) Ex.ejo.1).app.ee [[vo(x;))) () ue (x(x)) [.]]
(1)
FM is compatible with various interpolation paths (e.g., linear or Gaussian) and enables scaling CNFs to high-dimensional data while maintaining stable optimization. However, this objective is often intractable due to the need to evaluate the marginal px(x)) and vector field uz)x). Conditional Flow Matching (CFM) addressos this by introducing latent condition to index a family of probability paths. Instead of a single fixed pt, CFM considers a conditional density pr(2 =)) with associated vector field (z(z))). The overall path is recovered as Pe(x)= Ez (z) (x((2)))) The CFM objective is given by
(f)
[10o t) (1(
(2)
As it satisfies VoLCFM(0) - (O) under mild conditions, optimizing LCFM learns the correct global flow while sampling ouly from simpler conditional distributions. The key practical insight of CFM is to define the latent variable using the data samples themselves. In the standard independent coupling scheme, one sets 2 = (zo. 21), where Tu ~ Po(z) is sample from the source distribution and pi(x) is a sample from the target. The conditional path pa( 21) thus becomes a simple "bridge" between a single source point and a single target point, making its path and vector field trivial to compute. Consequently, the training process only requires the ability to sample from Po(x)) and pi (2),, removing any need to evaluate its probability density (or log-density). This provides immense flexibility in the choice of po-
With this simple framework, FM is able to model more flexible probability paths (Chen & Lipman 2024 Gat
et al. 2024 Stark et al. 2024 Cheng et al. 2025
Kapusniak et al. 2024] than diffusion models (Song el al.
2021a la Song & Ermon 2019 Song et al.
2021b
Rombach et al. 2022 Dhariwal & Nichol 2021p. FM has
also been extended to various domains, inc including image (Esser et al. 2024
Dao et al. 2023 Ren et al.
2024).
audio (Gann et al., 2024 Liu et al. 2023a Prajwal et al. 20244), video (Jin et al. 2025
Polyak et al.
2024
molecule (unnn
Koes
2024
Song et al. 2023). and text generation (Hu et al.
2024
2.2 Optimal Transport CFM (OT-CFM)
When trained with a globally optimal coupling, the flow map learned by FM aligns with the W22 geodesic between the source and target distributions. However, in practice, CFM typically adopts a simpler independent coupling lind (2o, 21) Po( (I(),, which pairs each source sample 2g Po with randomly chosen target sample I1 ~ Pi (I-CFM). Given any coupling q E I[(o: where IT(po)P)) denotes the set of all valid joint distributions with marginals Po and pr, the transport cost and the optimal transport plan are defined as:
C(q)= E 3 [||= ro 13]
arg min C(g). gEll(po.Pt)
The independent coupling Tind (o( (a1) generally incurs suboptimal transport cost. In this paper, we define the excessive cost relative to the optimal transport plan as the Wasserstein coupling gap:
Aw C(gima) C(T) 2 0.
(4)
This gap quantifies the degree of misalignment between the randomly paired paths and the true geodesic paths. A large gap forces the learned flow to exhibit unnecessary curvature, deviating from the straightest. possible transport map, which can in turn increase gradient variance and hinder training efficiency. To mitigate this issue, mini-batch optimal transport (BatchOT or OT-CFM) (Pooladian et al. 2023 Tong el al., 2024) selects, at each training step, the permutation that best matches the B source samples to the target samples, yiclding a locally optimal Wasserstein coupling. Because this permutation is recomputed for cach mini-batch, the aligument does not persist across iterations, which can limit long-term convergence beneffits.
2.3 Alternative Source Distributions
Target- Aprexxiaappp Source Distribution. TSFlow (Kollovich et al., 2025) extends target- approximating source distributions to time series forecasting, integrating Gaussian Process priors within CFM
4

---

Published in Transactions on Machine Learning Research (12/2025)
to align source distribution with temporal target data structure. HarmonicFlow (Stark et al. 2023 employs physics-based priors for protein-ligand docking through 3D geometric constraints, utilizing protein backbone torsion-angle distributions to preserve -Euiivaavirrr properties. AlphaFlow (Jing et al. 2024 learns directly from PDB structural databascs, replacing Gaussian noise with experimentally observed distributions to model biomolecular conformational variability. While promising, these approaches remain limited to simple domain-specific distributions and low-dimensional spaces.
Non-Gaussian Source in Cross-Modal Generation. CrossFlow (Liu et al. 2024 enables direct mapping from text to image embeddings using discrete language spaces as source distributions, overcoming traditional Gaussian space constraints. FlowTok (He et al. 2025) constructs a shared text-image token space by transforming Vision Transformer patch tokens into quantized discrete spaces, redefining image generation as 1D token sequence generation. However, these approaches require substantial engineering for source-target alignment, suggesting naive replacements may create more problems than solutions for complex high-dimensional data.
2.4
Von Mises-Fisher (vMF) Distribution
The von Mises-Fisher (vMF) distribution (Fisher 1953), denoted as vMF(u, x),, provides mathematically principled framework for this directional concentration, defining probability distribution on the unit hypersphere with mean direction and concentration parameter K. As increases, the distribution becomes more sharply concentrated around the mean direction μ.. When = 0, the distribution reduces to a uniform distribution over the unit sphere. This makes the vMF distribution particularly suitable for modeling directional data or sampling unit vectors with controlled angular concentration, which is useful in flow matching tasks where aligning the directionality of source and target samples is crucial.
3 Revisiting Gaussian Source Distributions for Flow Matching
The choice of the source distribution po is a fundamental element in flow matching (Lipman et al. 2023 Liu
et al., 2023b Tong et al., 2024). While the standard isotropic Gaussian distribution (N, is widely adopted for its simplicity and favorable mathematical properties, its optimality for complex image generation tasks remains open question. In this section, we point out geometric properties of Gaussian distributions, propose functionally equivalent directional sampling paradigm, and derive key insights for designing effective source distributions with 2D simulations.
3.1 High-Dimensional Geometry and Directional Decomposition
The slandard Gaussian Zg ~ 1) E R& serves as a conventional choice for the source distribution in generative models. In high-dimensional spaces, the vast majority of independent and identically distributed (i.i..)) Gaussian samples reside on a thin hyperspperical shell. Specifically, the Euclidean norm of zg follows x-distribution: ||=||| Xd). whose mean and variance are approximately /d- 1d2 and 1/2, respectively, This indicates that for sufficiently large d, most samples lie hypersphere whose radius is sharply concentrated in narrow band around 1/21 Motivated by this property, we propose the x-Sphere decomposition, directional decomposition scheme defined as:
ZG RJ rSG. with r r~> sG ~ U(S--))
where U(S=11) denotes the uniform distribution the unit sphere. Our X-Sphere decomposition preserves all Gaussian statistics while explicitly factorizing each sample into independent radius and directional unit vector (sce the derivation in Section A.. To verify that this thcoretical equivalence holds in practice, we further measure the FID scores of generated images using different combinations of training and inference source distributions. The result in Table [
indicates that the samples from x-Sphere and standard Gaussian distributions are interchangeable during both training and inference, without showing significant performance difference. Consequently, we regard these two sampling methods as functionally equivalent throughout this paper.
5

---

Published in Transactions on Machine Learning Research (12/2025)
In practice, normalized data-typically shifted to zero mean and scaled to lie within 1-1,1]-tend to concentrate within bounded norm region, forming thick hyperspherical shell. This enables a similar decomposition of data point = r181 E IR, where 81 E I2 is the unit vector representing its direction and r1 E R is the norm of the data sample. Under this formulation, we can reasonably mcasure the "angular similarity" between a sample from Gaussian and a data point using the cosine similarity, &G 81- Viewing the data distribution through this directional perspective reveals as a subset of directions of full hypersphere, a viewpoint we explore further in Section 4.2
3.2 Proposed Simulation for High-Dimensional Flow Analysis
While previous works have visualized flow matching between simple 2D distributions, there has been little attention to understanding the learning dynamics of flow models in high-dimensional settings. Davtyan et al. 2025 for example, illustrates 2D Gaussian to 2D Gaussian, shown in Fig. [ in Fig. 1), visualizes the transport paths from the 8 Gaussians to moons. These settings, however, fail to reflect the geometric properties of source and target samples and do not capture the behavior of flow matching in high-dimensional settings. To address this gap and analyze the behavior of flow matching models in high-dimensional settings, we propose a novel flow simulation setup that is more suitable for capturing the geometric properties uniquely observed in high-dimensional distributions.
a)) Tong et al. (2024). illustrated
Specifically, we simulate high-dimensional geometric structure in 2D by sampling random directions 0 E [0, 2T) and scaling with norms from x(d) distributions (o Source) following the x-Sphere decomposition, illustrated in Fig. [c). When analyzing flow matching with the proposed density-approximaaed sources and directional sources introduced in Section [ we normalize their mean norms to match the x-Sphere radius. This normalization allows us to observe transport path patterns comparable to the x-Sphere while ensuring similar geometric properties across all source types. For the target data ( Data), we sample points along three clusters with varying densities, reflecting the common scenario where real high-dimensional datasets scaled to [] (e-g., CTFAR-10 with norm 27.2) typically have smaller norms than their corresponding Gaussian sources (55.4) and reside inside the concentration shell. We train flow matching models on these geometrically designed distributions and visualize their transport trajectories. Blue dashed lines (-- ODE trajectory) show successful paths from source ( Source) to generated samples (o Generated), while red dashed lines (-- Bad ODE trajectory) indicate failures. Here, we define generated samples to the nearest data point is within one unit.
based on whether the L.2 distance from
To complement our primary visual analysis, we report the Normalized Wasserstein (Balaji et al.) 2019); averaged over 10 runs, as auxiliary measure. This metric captures how closely the generated samples resemble the true data distribution, even when the data contains multiple modes with inbalanced proportions. Unlike Wasserstein distance, the Normalized Wasserstein is robust to differences in mode proportions and focuses on the structural similarity between distributions. Lower values indicate better alignment with the true distribution. Since our setup is a controlled simulation, this metric should not be interpreted in absolute terms. Instead, it is most meaningful when used for relative comparison within the same group of source distributions. Additional metrics and implementation details are provided in Section B
4 Understanding Learning Dynamics of FM: Insights from 2D Simulations
Using the simulation approach proposed Section 3.2 we conduct comprehensive experiments to verify the
source distribution strategics proposed in Scction 1 Although simplified, these experiments preserve the
key geometric structures and transport dynamics of nigh-dimensiooal settings. They allow us to empirically
Table 1: FID scores under different combinations of training and inference source distributions.
Training
Inference
Gaussian
x-Sphere
Gaussian
4.40
4.29
x- x-Sphere
4.45
4.42
6

---

Published in Transactions on Machine Learning Research (12/2025)
Norm. W 1.34
Norm. W 1.40
Norm. W 1.47
Source
Source
Generated
Generated
Generated
Data
Data
Data
(a)
(b)
(c)
Figure 2: Visualization of flow matching with density-approximated source. (a) OT-CFM with
Approximated Source (iter 200), (b) OT-CFM with Approximated Source (iter 6000), (c) OT-CFM with Appraximated Source (iter 10000). "Norm. W denotes normalized Wasserstein, where a lower value indicates better generation performance.
probe the core mechanisms behind flow matching, offering insights into what drives its success or failure across different conditions.
4.1 Density Approximation Strategy
Recent studies (Pooladian al. 2023 Tong et. al. 2024) have shown that improving the pairing between source and target distributions produces straighter probability flow trajectories, fewer sampling steps at inference, and expedited training. These improvements are largely attributed to reduced transport costs achieved through better alignment. Motivated by these findings, we hypothesize that shaping the source distribution ion to closely resemble the target distribution while preserving the improved reduce transport cost and improve generation performance.
might further
To test our hypothesis, we first train a flow matching model, which we refer to as the density approzimator, to transport samples from the x-Sphere Gaussian distribution toward the target data. distribution. The density approximator is trained for different numbers of iterations (specifically, 200, 6,000, and 10,000) to achieve progressively stronger approximations of the larget distribution, and at each stage, we use the samples generated by the partially trained model as the new approximated source distribution, denoted by Pu- To ensure a consistent visual comparison across these stages, we then scale cach ýu to match the average norm of the original x-Sphere, as interpreting the transport dynamics without this aligument is challenging, as illustrated in Fig. This approach allows us to observe how the progressively approximated source distribution Po acts new source distribution for flow matching.
Contrary to initial hypothesis that source distribution resembling the target would improve generation performance, we observe in Fig. 2 that more samples fall outside the data modes as the source approximates the target data. Also, higher Normalized Wasserstein is observed when the source distribution becomes increasingly similar to the target distribution. This indicates that the performance degrades in spite of a better density approximation between the source and the target, implying that more similar source distribution might hinder accurate mode coverage and generation quality. This counterintuitive outcome challenges our hypothesis about source distribution design in flow matching and necessitates careful analysis of the underlying mechanism.
Mode Discrepancy. Taking a deeper look, as the source density approximator gradually Iearns to match the larget distribution, the approximated source ýo concentrales samples around each data mode ( Data). Subsequently, the distribution adjusted to spread outward ( Source), aligning its norm with that of the S-Sphere source, as discussed Scction 3.2. Comparing Fig. a)) (b) and (c), the density approximator reflects the two dominant data modes while gradually departing from the general spherical shell structure of the x-Sphere. However, po fails to capture the lower-left. mode in sparse density regions even after 10,000
7

---

Published in Transactions on Machine Learning Research (12/2025)
Norm. W 1.03
Norm. W 1.15
Norm. W = 0.52
Source
Generated
Generated
Generated Data
Data
Data
(a)
(b)
(c)
Figure 3: Visualization of flow matching with ideally aligned directional source. (a) OT-CFM with directional source, (b) OT-CFM with tight directional source, (c) global OT Pairing with directional source. 'Norm. W denotes Normalized Wasserstein.
training iterations. These 2D simulation results reveal fundamental problem that occurs in practice: any approximated distribution Pu inevitably suffers from information loss, particularly in low-density regions where several modes may be underrepresented omitted. This inaccurate approximation creates pairing mismatch where certain target data points I1 from underrepresented or omitted modes cannot be effectively paired with appropriate source samples 2g from the approximated source Pu- We term this phenomenon mode discrepancy- Consequently, even optimal transport pairing is forced learn inefficient and complex trajectories, potentially increasing the learning difficulty instead of reducing This analysis revcals that mode discrepancy poses fundamental limitation density approximation strategies for flow matching source distributions.
4.2 Directional Alignment Strategy
Recognizing the critical importance of preserving all data modes during approximation, we shift our focus toward different approach that sacrifices Specifically, we leverage x-Sphere decomposition and focus the cosine similarity between the source unit vector and the larget unit vector $1- In an ideal scenario where we know all data mode directions, we can incorporate this knowledge into our source modeling. For each data point belonging to a specific mode direction, we construct a x-Sphere source that maintains a cosine similarity above a threshold ? with the data (f.e., ag 81 2 7). This approach allows us to avoid the mode discrepancy problem encountered earlier. This donactional alignment approach visualized under our experimental setting in Fig. 3 where the source successfully reflects all data mode directions. Combined with the improved pairing strategy, we expect this configuration to yield optimal results.
information to better approximate data mode directions.
Imperfect Pairing. However, even when using a source distribution that covers all data modes through directional aligument and employing matc- OT pairing method, the configuration fails to achieve the expected optimal performance. As shown in Fig. 3(a) and (b), we observe that points are not generated perfectly within each data cluster some red points scatter outside the blue target clusters. This suboptimal behavior contrasts with the global OT-paired directional alignment source in Fig. 3(c), where all generated points successfully converge to their respective data clusters. These findings highlight the limitation of mini-batch OT pairing that still falls significantly short of perfect global OT pairing.
Path Entanglement. In addition, interestingly, we observe that using more tightly-aligned directional source by increasing 7 in the directional source alignment act ually leads to performance degradation ion, as shown in Fig. b(). From a geometric similarity perspective, as y approaches 1, Fig. 3(b) shows that the source distribution bocomes increasingly concentrated, spreading only as much as the data itself. However, we find that this reduction in source support adversely affects the overall performance. The underlying cause of this phenomenon be seen in Fig. 3 b)) where paths from each source to target cluster become heavily entangled
8

---

Published in Transactions on Machine Learning Research (12/2025)
Norm. W = 1.45
Norm, W = 1.50
Norm. W = 1.36
Source
Source
Source
Gerierated
Genierated
Getierated Dota
Data
Data
(a)
(b)
(c)
Figure 4: Visualization of flow matching with different methods. (a) I-CFM with Gaussian source (b)
OT-CFM with Gaussian source (c) I-CFM with prunod source.
in localized regions during generation. This excessive entanglement leads to highly inconsistent vector field directions within local areas, making the vector field difficult to learn and unstable during optimization.
To better understand the cause of this entanglement, we provide a mathematical analysis of how the geometry of source-target pairings changes under increased concentration of the source distribution. Consider two target points 24 within the same mode cluster, with their associated source points zo, zo drawn from the corresponding directional source cluster. Their linearly interpolated paths are given by 24 (1 f)zg + to and o/ = (1 - 1)=² + la respectively, with separation: d(l) It 2 () xx)) [(x1 - x(). This separation obeys ( l|d((]](]] I ||z0 z = O(11 - -/)1/)) Therefore, large values of 7 cause the trajectorics to be nearly coincident at initialization, which in turn Icads to a sharp increase in the required
local Lipschitz constant:
sin 9
(2) - 2n) T x/))
Liocal 1 - 111l + O - 7//22) where arccos
(6)
|||| - xo I1 ||
As a result, Local becomes larger, making the optimization increasingly unstable and slow. More detailed derivation is provided in Section
These findings demonstrate that without oracle-level pairing, learning becomes significantly more difficult due inconsistent vector fields induced by trajectory interference. To avoid this, the source distribution must retain the angular support sufficiently. The limitations observed here also extend to the density approximation discussed in Section 4.1 where increased approximation leads to reduced support, and imperfect pairing continues to pose challenges.
In conclusion, contrary to our initial expectation that combining mini-batch OT with improved source approximation (density or direction) would yield optimal results, our simulations reveal that increased approx- imation actually leads diminished performance due to mode discrepancy, imperfect pairing mechanisms, and path entanglement. This counterintuitive finding underscores the delicate balance required between source distribution design and coupling strategies in flow matching approaches.
4.3 Pairing Method Analysis
To understand how different pairing methods affect the learning dynamics of flow models, we further analyze the generation trajectories and learning patterns of each approach.
Independent Pairing (I-CFM). The generation trajectory of I-CFM with x-Sphere Gaussian source in Fig. ((a) reveals patterns that can be interpreted as a conceptually Iwo-stage transport process. In the initial stage, samples from the source move coarsely toward data-dense regions where the data modes are located. After this rough transport, fino-grained adjustments are performed in the second stage, where the paths are converged from all directions towards each mode. This occurs because the x-Sphere source distribution provides uniform coverage across all directions. When paired independently with the target data, each data
9

---

Published in Transactions on Machine Learning Research (12/2025)
High
Low
(a)
(b)
Figure 5: Visualization of flow trajectory heatmap. These heatmaps show the distribution of paths learned by each model. Color intensity corresponds to trajectory density, where brighter regions indicate higher concentrations of paths. (a) I-CFM and (b) OT-CFM.
sample receives supervision from sources distributed omnidirectionally, enabling comprehensive vector field learning around cach mode. This is clearly observed in Fig. 5(a), which shows the density of trajectories actually learned during training with independent pairing. The same principle applies to high-dimensional Gaussians, which also provide uniform directional coverage.
Improved Pairing (OT-CFM). In contrast, the OT-CFM trajectory heatmap in Fig. 5 () shows a markedly different pattern. OT-CFM uses a batch-level optimal transport pairing stralegy that minimizes the transport cost within each batch. This in-batch optimal pairing creates a more localized pairing where most source samples are matched with the ncarby target cluster. Consequently, the model Iearns denser and more linear trajectories between optimally paired source and target samples. This can be seen in Fig. [bb, where many source samples follow straighter paths compared to the more curved trajectories in Fig. (aa produced by I-CFM.
However, this improved path efficiency comes with a drawback. Because OT-CFM consistently pairs samples with nearby targets, the model learns the vector field primarily along narrow, cone-shaped directions centered around each data mode. As a result, it fails to learn the omnidirectional vector fields near the modes, which I-CFM captures better. This leads to insufficient learning of vector fields in broad angular directions, especially from the origin to the data modes, as shown in the trajectory heatmap in Fig. B
If all pairings were perfect, this focused learning approach would be both efficient and effective, the model could learn the vector field accurately and consistently along these specific trajectories. In practice however, the pairing is inevitably imperfect. When samples need to traverse these undertrained regions due to suboptimal pairings, they lack proper vector field guidance. Consequently, those samples often stall or veer off course, failing to reach the larget data. This failure is clearly illustrated in Fig. 4bb,, where multiple red points appear stranded midway, outside the data modes. This reveals a fundamental trade-off in OT-CFM: improved path efficiency comes at the expense of the robustness provided by omnidirectional coverage.
Low-density Directions. A common observation in both Fig. 4 (a) and (b) is that when samples are initialized in low-density directions-drrections where data is absent and the angular separation from data modes is large they often fail to reach the data manifold during generation. These failed trajectories appear as clusters of red dashed lines (--) near this direction. This issue arises because, interpolation paths from directions with sparse or absent data are sampled relatively less frequently during training, leading to lower training frequency for these regions. This is clearly visible in Fig. 5 where darker regions (such as the left and lower left areas) indicate lower training frequency compared other parts of the heatmap. The rarily of path traversal during training results in insufficient supervision for the velocity field in these regions. Consequently, the vector field in such areas becomes inaccurately learned, increasing the likelihood of generation failures from samples initialized in low-density directions.
Building upon these insights, we experiment with a simple strategy that reflects the flow learning dynamics observed in our simulation. By comparing the two pairing mothods, we find that learning an omnidiroctional vector field around each data mode leads to a more robust velocity field. We also observed that, across all pairing strategies, source samples from low-density directions tend to produce poor generations. Combining
10

---

Published in Transactions on Machine Learning Research (12/2025)
Table 2: Comparison of Approximated Target Distribution. Unconditional generation results on
CIFAR10 using different source distributions. All models are trained with OT-CFM
(Tong et al. 2024
Gaussian
DCT-Weak
DCT-Strong
GMM-1
GMM-2
GMM-10
CNF
FID 4
4.40
4.20
4.39
11.75
12.49
12.11
17.18
these findings, we train I-CFM using the full x-Sphere source to encourage robust and omnidirectional learning. Then, during sampling. we prune source samples from low-density directions to avoid poorly trained regions. This hybrid strategy, which is visualized in Fig- results in significantly fewer failed trajectories and improved performance by maintaining comprchensive training coverage while guiding sampling toward more reliable paths during generation. Through this approach, we gained valuable insights that could help identify better source distributions for flow matching models.
5 Empirical Validation on High-Dimensional Image Datasets
In this section, we extend our analysis high-dimensional image datasets through parallel experiments that validate the hypotheses and findings from Section We investigate whether the insights from our low-dimensional simulations transfer to real-world image generation tasks.
5.1 Density Approximation Strategy
In Section 4.1 we examined the intuitive hypothesis that using source distributions that better approximate the target distribution's density would improve the efficiency and performance of flow matching through our 2D simulations. Our findings revealed that density-approximated sources introduce mode discrepancy issues that hinder effective training and yield inferior performance compared to baseline. To evaluate whether this conclusion in 2D simulation can be generalized to high-dimensional real-world datasets, we investigate three progressively expressive approximations to the target distribution and assess their impact on flow matching performance and training dynamics. To evaluate progressively stronger approximations to the target distribution, we employ the following density approximation methods:
Discrete Cosine Transform (DCT) Ahmed et al. 1974 Strang 1999), commonly used in image compression (e.g., JPEG (Wallace 1992 separates spatial frequencies of the image, discarding high frequency components that are less perceptible to humans (Wang al.) 2004). Inspired by this, we apply DCT filtering to Gaussian source to remove less prominent high-frequency components of the target. distribution. See Section D for more details.
Gaussian Mixture Model (GMM) (Reynolds et al. 2009 approximates a complex distribution by fitting multiple Gaussians. We train GMMs with 1, 2, and 10 components using the Expectation- Maximization (EM) algorithm (Moon) 1996), and use these as alternative source distributions.
Continuous Normalizing Flow (CNF) (Grathwohl et al.] 22018] Chen et al. 2018) constructs flexible distributions by continuously transforming a simple base density through a series of invertible mappings parameterized by neural ODFs. Although its invertibility constraint might limit expressiveness, CNF still serve as sophisticated approximation of the data distribution. We train FFJORD (Grathwohl et al. 2018h, popular CNF model, and use ils output as the source distributions for flow malching.
As shown in Table 2 while DCT-based mild refinement improves performance compared to the Gaussian baseline, stronger approximations to the target distribution progressively degrade performance (DCT GMM > CNF in terms of quality). This finding aligns with our simulation results from Section where we observed similar performance degradation with stronger density approximations.
Although high-dimensional settings make it difficult to analyze the specific underlying mechanisms, the consistent trend across both simulations and real experiments suggests a fundamental issue with density-basod approximation strategics. Drawing from our 2D analysis, a key insight is that approximating the target distribution inevitably leads to information loss, particularly in low-density regions. These regions tend
11

---

Published in Transactions on Machine Learning Research (12/2025)
Table 3: Directional Alignment Performance. FID scores on CLFAR10 using Euler integration for 100
NFE, comparing standard Gaussian source with directionally-aligned alternatives.
Oracle-vMF
Kmeans-vMF
Gaussian
Algorithm
R == 00
R == 3000
A 1500
A 1500
900
A =300
K 100
K = 50
(N 0)
OT-CFM
0.74
1.98
2.86
7.11
5.86
4.64
4.22
4.15
4.40
to be underrepresented or omitted in the support of the approximate source po. As demonstrated in our 2D simulation (Fig. Pb),, when no source sample z0 ~ po directionally aligned with given target sample æ ~ Pt (such as samples from sparse modes), even OT-based coupling produces inefficient, entangled trajectories that increase training complexity and hurt generative performance.
5.2 Directional Alignment Strategy
In our previous analysis, we identified a fundamental limitation: approximating the target distribution's density incvitably leads to information loss, and using such approximations as source distributions results in mode discrepancy, where certain data modes lack corresponding suitable samples, hindering elfective training. To address this challenge, we propose directional alignment strategy that focuses on comprehensive mode coverage, paralleling our 2D simulations in Section 4.2
We find that the FID between normalized and original data is only 0.29, demonstrating that directional information is the key factor for generation quality. Based on this insight, rather than attempting to match the complete target density which suffers from information loss sparse regions we focus on preserving directions where target data exists while removing directions without data. This approach prioritizes directional information of data modes while ignoring norm information, thereby avoiding the mode discrepancy problems inherent in density approximation strategies.
To obtain a distribution aligned with target directions, we leverage the X-Sphere decomposition introduced in Section 3.1 and vMF distribution. This allows us to approximate directional pairing by generating source samples Tp that are directionally aligned with each target sample ø1. The specific sampling stratogy follows:
To = r with ro x(d), 80 MF((((1)
(7)
where the vMF's mean direction ((x1) aligns the direction of zo with 21, while the probabilistic radius r and concentration parameter s(x1) enable probabilistic rather than deterministic pairing, providing robust alignment through controlled variability.
5.2.1 Oracle Approach
To investigate the potential performance when sampling focused on directions where data actually exists, we design an oracle pairing experiment. In this oracle sccnario, the vMF mean direction is set to each normalized data point, and varying values control the tightness of pairing between source and target samples. This experimental setting closely corresponds to the ideal scenario illustrated in our 2D simulation (Fig. 3c)). When к approaches infinity, 2o and Ii become directionally identical, reducing the task to simple norm matching. Conversely, when K = 0, the distribution becomes equivalent to Gaussian according to Section 3.1
As shown in Table g. while x approaching infinity yiclds FID scores near zero, this result is trivial since it merely reproduces the training data with norm scaling. More importantly, at reasonable k values (1500, 3000), we observe that multiple generations from the same vMF(u,, k) distribution produce images that are structurally similar yet vary in fine details. This demonstrates that the model learns to generate diverse samples around each data point rather than simply memorizing individual training examples (see Fig, 9 in appendix). Even under oracle conditions, the model generates novel images not present in the original distribution, confirming the potential of directional alignment strategies.
However, this approach has a critical limitation: it requires storing the entire training dataset generate corresponding direction-aligned source samples, making it impractical for large-scale applications.
12

---

Published in Transactions on Machine Learning Research (12/2025)
5.2.2 Clustering-basee Approach
The oracle pairing method, while effective, is impractical due to its massive storage requirements. To address this, we propose a more scalable alternative: clustering-based pairing. This method approximates the oracle setup by grouping similar data points and assigning them a shared source distribution, drastically reducing the computational and storage overhead.
The process is straight.orward. First, we partition the normalized target data into K groups using spherical K-means clustering [MacQueen] 1967). For each of the K clusters, we then construct a vMF distribution as vMF(c,/lle, k), where Cj is the centroid of cluster j, and sample source points following Eq. 7 Each target sample x1 then paired with a source sample zy drawn from the vMF corresponding to its assigned cluster. While this pairing is suboptimal compared to the oracle pairing, it offers practical compromise between random Gaussian pairing and perfect directional alignment for suitable choices of K and K.
In Table 3 we benchmark the performance across range of concentrations k, fixing K 3 found by an elbow study with silhouette scores. As the concentration parameter x decreases, each distribution covers a broader angular range. When x = 0, the distribution becomes nearly identical to Gaussian, and generation performance is correspondingly similar. For moderately concentrated ranges of x E {50, 100), we observe notable performance improvements. At high concentrations (m 2 300), however, performance drops below that of the Gaussian bascline. This degradation, similar to what we observed in Fig. ()), stems from path entanglement due to support deficiency as discussed in Section 4.2 When the source distribution ion becomes too concentrated, trajectories from nearby source points become entangled, making the vector field difficult to learn and causing training instability.
This finding highlights a fundamental trade-off: while directional alignment offers advantages, the source distribution must retain sufficient angular support to avoid trajectory interference. Without oracle-level pairing that guarantees consistent and perfect alignment, overly concentrated distributions create more problems than they solve, confirming that robust flow learning requires balanced coverage rather than extreme concentration.
6 Proposed Method
In this section, introduce two complementary strategies based previous findings. We first propose "Pruned Sampling". which creates a more efficient souree distribution by identifying and removing directions that are directionally irrelevant to the target data. We then introduce "Norm Alignment" to resolve the scale mismatch between the source and target distributions. By creating a data-informed source, these methods improve the performance and stability of flow matching models without requiring architectural changes or retraining.
6.1 Pruned Sampling
Our previous experiments showed that we need to maintain sufficient support, but is difficult to determine how much support is needed, Instead of attempting to find this amount, we propose the opposite strategy: pruning the Gaussian. The core idea is to start with Gaussian source that covers all directions, then identify and prune away regions that are directionally irrelevant to the target data. By removing directions where data is absent, we can adjust the source distribution to better align with the target manifold.
While several approaches can be employed to identify directions where data is absent or sparse, we utilize Principal Component Analysis (PCA) (Pearson 1901h for simplicity. Specifically, given a dataset D = (a C Rdd we first Le-normaliie each sample by n. = x,/ = and compute an orthonormal basis V = - va] E Rdxd via PCA, where the basis vectors are ordered by their corresponding eigenvalues in descending order. To account for directional symmetry, construct extended basis by including the opposite directions:
Vext [u1, Va. Ed+l. U2al E IR dxd2
where Uk-d -t for k d 1..... ..... 2d. For every basis vector By E Vext, we measure the largest cosine similarity 7k max;(ux, F..) for k 1,....2 2d. Directions with low Ta indicate that they are remote from
13

---

Published in Transactions on Machine Learning Research (12/2025)
Table 4: Pruned Sampling. Comparison of FID scores for 1-CFM and OT-CFM under different training and inference source confiurattonss on CIFAR10. Euler integration is used for all evaluations. The best scores are indicated in bold.
Method
Train Source
Inference Source
FID (NFE=5)
FID (NFE=100)
Ganssian
Ganssian
34.74
4.36
I-CFM
Pruned
Pruned
34.71
4.17
Gaussian
Pruned
28.99
3.95
Ganssian
Ganssian
18.55
4.40
OT-CFM
Pruned
Pruned
20.47
4.18
Gaussian
Pruned
17.29
4.10
the data distribution. We denote set of such basis vectors R {k E (l,...,,4 nh <T), where is hyperparameter indicating the threshold. We then apply rejection sampling: we sample 2o (O, and retain it if and only if maxKeR Uk, Teall prevent potential support deficiency in critical directions. Detailed experimental scttings can be found Sections and
Fi S Tri where > provides a more conservative margin to
Based on this approach, we compare three strategic approaches: (i) Gaussian where the model is trained and sampled from the complete Gaussian distribution; (ii) Pruned->Prune which involves training and sampling exclusively from directions that meet the pruning criteria; and (iii) Gaussian-PPruned where the model is trained on the complete Gaussian distribution but sampled only from its pruned subset.
Empirical results in Table 4 reveal unexpected patterns. The fully pruned method (ii) yields inconsistent performance compared to the Gaussian baseline (i). While it offers marginal improvements of function evaluations (NFE), it has minimal impact on I-CFM at low NFE and negatively affects the performance of OT-CFM. In contrast, the hybrid method (iii consistently outperforms the other methods across all evaluated configurations.
high number
Although this observation may seem counterintuitive it aligns with our 2D simulation findings in Section The effectiveness the hybrid CCaussian-PPrueel strategy stems from the distinct requirements of the training and inference phases.
During training, leveraging the full Gaussian source is advantagoous. A high-dimensional Gaussian distribution provides uniform angular coverage, ensuring the model learns the vector field omnidirectionally around data modes. This comprehensive training is critical for building a robust model that can generalize across diverse initialization conditions. Conversely, restricting training to only the pruned directions (8) limits the model's exposure. Without supervision from the excluded directions (Sa-1 \ (), the model's ability to generalize to unseen trajectories is compromised. This limitation is particularly detrimental at low NFEs, where the model has fewer opportunities to correct for poor initializations.
During inference, however, certain regions of the Gaussian source are suboptimal for sampling. These include areas devoid of data and singular points equidistant from multiple data modes. Source samples (ro) from these regions are surrounded by fewer interpolation paths, leading to less frequent updates during training (see Fig. 5) Consequently, the vector field near these samples is learned less accurately, making them prone to generating erroneous outputs. Pruning these directions from source distribution during inference effectively guides the sampling process toward more reliable paths, enhancing generation quality.
In summary, the optimal strategy involves training on the complete Gaussian distribution ensure the model learns comprehensive and generalizable vector field, while sampling exclusively from the pruned subset to avoid regions where the learned field is likely to be inaccurate.
This finding has important practical implications. Pruned Sampling can be applied as a post-processing to any pretrained flow matching model that uses a Gaussian source distribution, requiring no retraining or architoctural modifications. We demonstrate the effectiveness and scalability of our approach across multiple settings. In Table 5 applying Pruned Sampling a sigher-dimensions ImageNet64 pretrained model yields consistent performance improvements, confirming the scalability of approach. Furthermore, Table G
14

---

Published in Transactions on Machine Learning Research (12/2025)
Table FID scores on ImageNet64 using Euler integration for OT-CFM w/ //w source Pruned.
NFE
Model
Source
5
10
20
100
Gaussian
53.95
18. 88.84
10.62
9.10
OT-CFM
Prumed
49.56
16.70
9.54
8.78
Table 6: FID scores on CIFAR10 using Fuler integration for various numbers of function evaluations (NFE) for OT-CFM and I-CFM, comparing the baseline (Gaussian source) with our proposed method that incorporates Pruned and/or NormAlign. Our methods are indicated in ilalics. Reported values are mean + standard deviation.
NFE
Train Method
Source
20
100
OT-CFM (Baseline)
Gaussian
19.80 0.39
10.95 + 0.34
7.41 0.20
5.61 0.10
4.40 + 0.01
OT-CFM
Pruned
17.24 + 0.08
9.17 + 0.14
6.34 = 0.05
4.92 - 0.11
4.15 + 0.07
OT-CFM + NormAlign
Gaussian
24.92 0.17
11.39 + 0.01
6.81 = 0.07
4.98 0.06
4.03 + 0.08
OT-CFM + Normallign
Pruned
21.52 0.22
9.55 + 0.06
5.85 = 0.03
4.53 0.06
3.88 + 0.03
I-CFM (Baselinc)
Gaussian
34.49 0.24
13.30 0.06
7.75 = 0.08
5.63 0.06
4.35 0.02
I-CFM
Pruned
28.78 + 0.19
10.37 = 0.15
6.40 = 0.06
4.89 - 0.09
3.97 + 0.03
I-CFM+ NormAlign
Gaussian
52.20 0.09
18.94 + 0.18
8.10 0.04
4.93 0.08
3.79 + 0.02
I-CFM NormAligg
Proned
48.10 0.34
16.58 + 0.24
7.04 = 0.06
4.56 0.03
3.64 + 0.02
shows that Pruned Sampling consistently enhances performance for both OT-CFM and I-CFM, regardless of the number of function evaluations (NFE). By simply filtering out initialization points from data-sparse regions during inference, existing models can achieve immediate performance improvements with minimal computational overhead.
6.2 Norm Alignment Strategy
In addition to the directional considerations, we identify another fundamental disparity in flow matching: the significant difference in scale between the source and target distributions. Flow matching models conventionally operate between standard Gaussian source /00 7) and the target data distributions, creating substantial norm disparity that impacts model performance. The standard Gaussian source samples typically exhibit norms of approximately vd, while target data samples (often normalized to [-1,1]d) possess norms cousiderably smaller. This disparity necessitates significant compulational resources either in model capacity or training iterations resolve, potentially diverting attention from more crucial aspects of distribution estimation.
To address this disparity, we propose Norm Alignment strategy. Specifically, we compute the expected norm of the target distribution, Ez [|]=]]], and that of the source distribution, Ezooppo [[]=ol]], which is F[((d)) when Po = (0, Z). We scale the target samples to z = x E [|]]== [I], placing the transformed target distribution pl on a hypersphere with radius matching the average norm of po- During inference, we reverse this scaling by multiplying x1 by F|] thus recovering the original scale of the target samples.
Aligning the average norms of source and target samples through the proportional scaling described above yields substantial performance gains, as demonstrated in Table This suggests that flow models face greater difficulty than expected in learning to resolve norm mismatches, requiring significant computational resources that could otherwise be devoted to more essential aspects of the generation task. However, at low numbers of function evaluations (NFE), applying Norm Alignment can actually degrade performance. This because, when both source and target distributions lie on the same norm hypersphere, some transport trajectories are restricted to move close the surface of this hypersphere. As result, the transport paths become more curved, requiring a greater number of NFEs for the model to accurately follow these paths and achieve effective transport. In contrast, without norm alignment, source samples move directly toward
15

---

Published in Transactions on Machine Learning Research (12/2025)
lower-norm regions near the data manifold, resulting in straighter transport paths at low NFEs. A more detailed discussion and visualization of this phenomenon can be found in Section
As a final remark, combining Norm Aligument and Pruned Sumpling at NFE 100 on CIFARIO dataset yields substantial improvements: reducing FID by 0.67 for OT-CFM and 0.72 for I-CFM. These results highlight the importance of addressing both directionality and norm alignment in the source distribution to fully unlock the potential of flow matching models.
7
Conclusion
This work investigates whether alternative source distributions can outperform the standard Gaussian in flow matching. To better understand how flow matching models learn, we propose novel 2D simulation designed to visualize and interpret high-dimensional behaviors in flow matching. Our simulations show strong agreement with actual high-dimensional results, highlighting the effectiveness of this approach for analyzing and gaining insights into flow matching.
Our analysis reveals that intuitive strategics often fail. Density approximation approaches, using models like GMMs or CNIs, suffer from mode discrepancy: by omitting low-density target modes, they force the model learn inefficient pathways. Similarly, directional alignment methods induce path entanglement when the source becomes too concentrated, destabilizing optimization. We find that the success of the Gaussian distribution lies in its omnidirectional coverage, which ensures robust vector field learning around all data modes.
Building on these insights, we introduce a hybrid framework that combines Norm Aligmment during training with Pruned Sampling at inference. This preserves the robustness of Gaussian-based training while climinating problematic initializations from data-sparse regions. Crucially, Pruned Sampling can be applied to pre-trained models without any retraining, offering a significant, practical advantage. Our extensive experiments confirm consistent improvements across multiple settings, demonstrating both the practical value of our insights and the effectiveness of our readily applicable technique for advancing flow matching.
Acknowledgment
This work was supported by Youlchon Foundation, NRF grants (RS-2021-NR05511, RS-2024-00336576, RS-2023-0022663) and ITTP grants (19-2022-112200644 RS-2024-00033111) by the government of Korea.
References
N. Ahmed, T. Natarajan, and K.R. Rao. Discrete cosine transform. IEEE Transactions on Computers, C-23 (1):90-93, -3,009,,1110 1974. doi: 10.1109 20199/CCC.174.22377777
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden, Stochastic interpolants: unifying framework for flows and diffusions. Xi preprint arXin:2303.08700 2023.
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of deep generative models. arXiv preprint arxiv:1710. 11379, 2017.
Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Normalized wasserstein for mixture distributions with applications in adversarial learning and domain adaptation. In ICCV, 2019.
Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang. Ze Wang, Zicheng Liu, Difan Zou, and Bhiksha Raj. Masked autoencoders are effective tokenizers for diffusion models. In International Conference on Machine Learning (ICML), 2025.
Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. arXiu:24 10733, 2024.
Ricky TQ Chen and Yaron Lipman. Flow matching on general geometries. In ICLR, 2024.
16

---

Published in Transactions on Machine Learning Research (12/2025)
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In NewrIPS, 2018.
Chaoran Cheng, Jiahan Li, Jiajun Fan, and Ge Liu. a-flow: unified framework for contimuous-state discrete flow matching models. arxie preprint erx:25504. 10283, 2025.
Jacwoong Choi, Junho Lee, Changycon Yoon, Jung Ho Park, Geonho Hwang, and Myungjoo Kang. Do not escape from the manifold: Discovering the local coordinates the latent space of gans. arXiv preprial 0rXiv:2106.0630 2021.
Quan Dao, Hao Phung, Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint arXiv:2307.08630 2023.
Aram Davtyan, Leello Tadesse Dadi, Volkan Cevher, and Paolo Favaro. Faster inference of flow-based generative models via improved data-noisc coupling. In ICLR, 2025.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fci. Imagenet: A large-scale hicrarchical image database. In 2009 IEEE Conference on Compuler Vision and Pallern Recognilion, pp. 248 255, 2009. doi: D0.109/CCVR..200552202
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
lan Dumn and David Ryan Koes. Mixed continuous and categorical flow matching for 3d de novo molecule generation. ArXie, pp. arXiv-2404, 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Miller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning (ICML), 2024.
Ronald Aylmer Fisher. Dispersion on sphere. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 217((130):295-33 305, 1953.
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky TQ Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. Discrete flow matching. In NIPS, 2024.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. ICLR, 2018.
Wenhao Guan, Kaidi Wang, Wangjin Zhou, Yang Wang, Feng Deng, Hui Wang, Lin Li, Qingyang Hong, and Yong Qin. Lafma: A latent flow matching model for text-to-audio generation. arXiv preprint 0rXiv:2406.08200 2024.
Jaehoon Hahm, Junho Lee, Sunghyun Kim, and Joonseok Lee. Isometric representation learning for disentangled latent space of diffusion models, arXiv preprint arXiw:2407.11451, 2024.
Ju He, Qihang Yu, Qihao Liu, and Liang-Chie Chieh Chen. Flowtok: Flowing seamlessly across text and image tokens. arXiv preprint ar arx::2200. 10772, 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NewrIPS, 33:6840 6851, 2020.
Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Björn Ommer, and Cees Snoek. Flow matching for conditional text generation in few sampling steps. Tn Proceedings of the 1 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 380-392,00 2024.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. In ICI.R, 2025.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Bowen Jing, Bonnie Berger, and Tommi Jaakkola. Alphafold meets flow matching for generating protein ensembles. In ICML, 2024.
Kacper Kapusniak, Peter Potaptchik, Teodora Reu, Leo Zhang, Alexander Tong, Michael Bronstein, Joey Bose, and Francesco Di Giovanni. Metric flow matching for smooth interpolations on the data manifold. In NIPS, 2024.
Marcel Kollovieh, Marten Lienen, David Lüdke, Leo Schwinn, and Stephan Günnemann. Flow matching with gaussian process priors for probabilistic time series forecasting. In ICLR, 2025.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://www.et toronto...... toronto..du//kkr......
featusng--atuure-22000
Junho Lee, Jcongwoo Shin, Hyungwook Choi, and Joonscok Lec. Latent diffusion models with masked autoencoders. arXiv:2507.09984 2025.
Sangyun Lee, Beomsu Kim, and Jong Chul Ye. Minimixing trajectory curvature of ode-based generative models. In ICML, 2023.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modcling. In ICLR, 2023.
Alexander H Liu, Matt Le, Apoorv Vyas, Bowen Shi, Andros Tjandra, and Wei-Ning Hsu. Generative pre-training for speech with flow matching. arXiv preprint arXiv:2310.16333 2023a.
Qihao Liu, Xi Yin, Alan Yuille, Andrew Brown, and Mannat Singh. Flowing from words to pixels: A framework for cross-modality evolution. arXiv preprint 2rXiv:2412.15211 2024.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023b.
James MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposiun Mathemalical Stalistics and Probabilily, Volume 1: Slatistics, volume 5, pp. 281-298. University of California press, 1967.
Todd K Moon. The expectation 1996.
expectation-maxxiiizttie maximization algorithm. IEEE Signal processing magazine, 13(6):47-60,
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021.
Karl Pearson. Principal components analysis. The London, Edinburgh, and Dublin Philosophical Magazine and Jourraal of Science, 6(2):559, 1901.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiu:2307.01952. 2023.
A Polyak, A Zohar, A Brown, Tjandra, A Sinha, A Lee, Vyas, B Shi, CY Ma, CY Chuang, et al. Movie gen: cast of media foundation models, 2025. arriv:2410.13790 2024.
Aram-Ales Alexande Pooladian, Heli Ben- Ben-Hamu, Carles Domingo-Enrinn Enrich, Brandon Amos, Yaron Lipman, and Ricky TQ Chen. Multisample flow matching: Straightening flows with minibatch couplings. In TCML, 2023.
KR Prajwal, Bowen Shi, Matthew Lee, Apoorv Vyas, Andros Tjandra, Mahi Luthra, Baishan Guo, Huiyu Wang, Triantafyllos Afouras, David Kant, al. Musicflow: Cascaded flow matching for text guided music generation. arXiv preprint ar 2xx.:2410.204-: 2024.
18

---

Published in Transactions on Machine Learning Research (12/2025)
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakorn. Diffusion autoencoders: Toward meaningful and decodable representation. In Proceedings of the TEEE/CV conference on computer vision and pattern recognition, pp. 10619 10629, 2022.
Sucheng Ren, Qihang Yu, Ju He, Xiaohui Shen, Alan Yuille, and Liang-Chich Chen. Flowar: Scale-wise autorogressive image generation meets flow matching. arXiv preprint arXiv:2412. 15205, 2024.
Douglas Reynolds et al. Gaussian mixture models. Encyclopedia of biometrics, (41(669-633).3, 2009.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resoltionn image synthesis with latent diffusion models. In CVPR, pp. 10684 10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention- 2015: 18th internalional conference, Munich, Germany, Oclober 5-9, 2015, proceedings, parl. III 18, pp. 234-241 Springer, 2015.
Jascha Dohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya. Ganguli. Deep unsupervised learning using noncquilibrium thermodynamies. In ICML, pp. 2256-2265. pmlr, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurIPS, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021b.
Yuxuan Song, Jingjing Gong, Minkai Xu, Ziyao Cao, Yanyan Lan, Stefano Ermon, Hao Zhou, and Wei- Ma. Equivariant flow matching with hybrid probability transport for 3d molecule generation. Advances in Neural Information Processing Systems, 36:549 568, 2023.
Hannes Stark, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Harmonic prior relf-conditioned flow matching for multi-ligand docking and binding site design. In NeurIPS 2023 AT for Science Workshop, 2023.
Hannes Stark, Bowen Jing, Chenyu Wang, Gabricle Corso, Bonnic Berger, Regina Barzilay, and Tommi Jaakkola. Dirichlet flow matching with applications to dna sequence design. In ICML, 2024.
Gilbert Strang. The discrete cosine transform. SIAM review, 41(1):135 147, 1999.
Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguct, Yanlci Zhang, Jarrid Rector-Brooke Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. TMLRR 2024.
Gregory K Wallace. The jpeg still picture compression standard. IEEE transactions on consumer electronics, 38(1)xxvii-xxxx xxxiv, 1992.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Fero P Simoncelli. Image quality assessment: from visibility to structural similarity. IEEE transactions image processing, 13(4):600 612, 2004.
Siyu Xing, Jie Cao, Huaibo Huang, Xiao-Yu Zhang, and Ran He. Exploring straighter trajectories of flow matching with diffusion guidance. arXiv preprint orXim:2311.16007 2023.
Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma latent diffusion models, In CVPR, 2025.
19

---

Published in Transactions on Machine Learning Research (12/2025)
Appendix
A Derivation of the x-Sphere Decomposition
Let I) with density
f(z) = (2T)d/2 exp( 211x111)
Change of variables. Let us denote x = rs, where r = ||æ|| E (0) and s = >>/||>/12 E S The mapping (r, 8) - x one-to-one on (0, xi) x 5 with Jacobian determinant |det. J| ,d- giving the joint density
1 fn)(r,s)= = (2m) (2m)
12 12
r r> 0, s SE S
Marginal of s. Integrating frio over yields
fa(8) = (00 (r) dr (2m)d// 6 6-2222 dr
/
x0
Wd-1
because the integral equals to
where W 4/2 ) is the surface area of 5d-1, Henco, ~ U((d-1), where U((d-1) denoles the uniform distribution on the unit sphere.
Marginal of Integrating Jr,s over the sphere and using Ssa-1 ds wa-1 gives
f-(r)= = wa-1 24/2-1T(5) d 1
which is the probability density function of the Xa distribution.
Because fre factorizes as f+(r)f(ss), and independent. Therefore, $ rss with x(d) and 8G U(Su-1).
B Implementation Details and Results for 2D Simulations
Table 7: Performance Comparison in 2D Simulation. Quantitative evaluation of flow matching performance across different source distribution strategies and pairing methods in 2D simulations. Results are averaged over 10 independent runs with standard deviations. Lower values indicate better performance for all metrics.
Source
Method
AMD +
Failure Rate 4
MDD
Norm. Wasserstein +
Gaussian (x-Sphere)
I-CM
0.19 + 0.03
1.88 + 0.73
0.100 + 0.031
1.45 + 0.15
Gaussian (x-Sphere)
OT-CFM
0.29 0.05
4.35 1.60
0.066 + 0.028
1.50
Pruned Gaussian (x-Sphere)
L-CFM
0.18 0.04
1.40 4.9
0.128 1 0.056
1.312
Flow Model 200-iter
OT-CFM
0.31 0.09
4.99 2.322
0.056 + 0.014
1.34
Flow Model 6k-iter
OT-CFM
0.32 0.05
6.05 105:1.24
0.017 + 0.005
1.11
Flow Model 10k-iter
OT-CFM
0.32 0.02
6.14
0.020 + 0.009
1.17
Directional Align (Normal)
Perfect OT
0.16 0.01
0.93
0.001 + 0.001
0,52 0.005
Directional Align (Normal)
OT-CFM
0.18 0.02
1.80 0.51
0.016 0.008
1.03 0.22
Directional Align (Tight)
OT-CFM
0.26 0.02
4.08 + 0.68
0.012 + 0.004
1.15
Implementation Details. Our 2D simulations are implemented using PyTorch with the TorchCFM library [Tong et al., 2024). The vector field model employs a multi-layer perceptron (MLP) architecture with time-varying inputs, trained using the Adam optimizer for 20,000 itcrations with a batch size of 16. The network consists of fully connected layers with hidden dimensions carefully chosen capture the flow dynamics in 2D space.
20

---

Published in Transactions on Machine Learning Research (12/2025)
To simulate high-dimensionall geometric properties within our 2D framework, implement the x-Sphere decomposition as follows. For the source distribution, we sample radii from a chi distribution with d = 3072 degrees of freedom to approximate high-dimensional norm concentration, then uniformly sample directions E [0, 2T.. For the target distribution, we construct three directional clusters with varying densitics to reflect realistic data scenarios. The clusters are positioned al base angles [4.4956, 5.9167, 1.1018) radians with corresponding density ratios [0.05, 0.3, 0.65]. Each cluster is generated by sampling angles around the base direction with small perturbations (+100 plus additional Gaussian noise) and radii from (6225).
Trajectory visualization is performed every 5,000 training iterations using the Dormand- Prince ODE solver with adaptive step size control (absolute tolerance = 1 x 10-4, relative tolerance = 1 x 10-²). We classify trajectories as successful or failed based on the L2 distance between generated samples and the nearest target data point, using a threshold of 1.0 unit. Successful trajectories (distance & 1.0) are visualized in light steel blue, while failed trajectories (distance > 1.0) appear in indian red, providing clear visual feedback on the effectiveness of different source distribution strategics.
Evaluation Metrics. Average Minimum Distance (AMD) measures generation accuracy by computing the mean L2 distance between each generated sample and its nearest target data point. This metric directly quantifies how closely the generated samples align with the true data distribution, with lower values indicating better approximation quality. Failure Rate quantifies the proportion of generated samples that exceed a distance threshold of 1.0 unit from any target data point. This binary metric captures the frequency of catastrophic generation failures where samples fall completely outside the data manifold, providing insight into the robustness of different source distribution strategies. Mode Distribution Divergence (MDD) assesses structural fidelity by measuring the KT--ivergence between the mode distributions of generated and true data samples. We compute this by first assigning each sample to its nearest data mode cluster, then calculating the divergence between the resulting mode proportion vectors. This metric is particularly sensitive to mode collapse or imbalanced mode coverage, making it crucial for evaluating multimodal generation quality. Normalized Wasserstein quantifies the overall discrepancy between generated and true data distributions using the Wasserstein-1 distance, normalized by the average pairwise distance within the true data distribution. This normalization makes the metric robust to scale differences and enables meaningful comparison across different experimental configurations. Unlike standard Wasserstein distance, this normalized variant is specifically designed to handle cases where data consists of multiple modes with potentially imbalanced proportions, providing a more reliable measure of distributional similarity in complex, multimodal settings.
C Derivation of Path-EEtaaglement Bound
Setup. Let su, s Lid. Cap(u, 7) be unit-norm directions drawn from the spherical-cap distribution
Cap(a, 7) = { {sES TT 2 2}. 0 0<><1.
Writing 0 = (t sk),, we have
E $o sg 0(1 7)
B[Ø] = (1
(A.1)
lid. x(d) and set zo = r r z/ r' s/g
Bounding the initial Euclidean separation. Because E(r] z Vd, we have
Draw radii r,
1100 2'612 = ,² + 12 -2rr' So0 so xe 2d(1 so s6).
2rr's/
so that
E 2o 2'6ll
(A.2)
Along the linear interpolation. For the straight-line paths Iee = (1 - +)x0 txı, x/ (1 1)26 + txi,
let d(t) = 2, - z/.. Then
min (E
[d(||| x ||:00 - =a'l = ((11 - 1/21
(A.3)
21

---

Published in Transactions on Machine Learning Research (12/2025)
Implication for the local Lipschitz constant. A finite-difference estimate of the learned vector-field Lipschitz constant yields
Llocl
2
| 1) ro (0(( 21 - x'yll
sin 8
f cos -
( (A.4) | - zo
(21-20)
|o -
||| - न + a(1 - 7)1/2)²
As 7-,1, the (1 1)2 term dominates the denominator, so Liocal + so, capturing the path-entannlement effect.
DCT Refinement
We designed the DCT masks to filter out unnecessary high frequency details from both the source distribution and the target distribution.
D.1 DCT Block
The Discrete Cosine Transform (DCT) (Ahmed et al. 1974 Strang 1999 cosine basis functions with varying two-dimensional spatial frequencies. In practice, the 2D DCT is commonly applied to zero-centered image blocks of size x D, producing a corresponding x D DCT coefficient matrix B.
decomposes an image into set of
D-1D-1
(2r + 1)uT
(2y + 1)0z
Bud = agaa
Σ
Σ
byy cos
cos
16
16
-= y=
if = 0,
1
if 0,
DD
VD
Qu
0x+
2
if u
V D
2
if + O.
where x and denote the horizontal and vertical pixel coordinates, while u and v are indices of corresponding horizontal and vertical spatial frequencies. a serves as normalization term to ensure the orthogonality of the DCT basis functions. Following JPEG (Wallace 1992) compression standard, D 8 splitting the whole image into 8 8 blocks. Then, each blocks is converted to the YCICR color space, which separates image content into luminance channel Y, representing brightness, and two chrominance channels Cb and Cr, representing color information. Subsequently, the DCT is applied independently to each D) x D block within all three channels.
D.2 Data dependent DCT mask
To construct datasct-specific DCT-based frequency masks, first applied the Discrete Cosine Transform (DCT) to every D D block in the CIFAR-10 dataset. We computed the mean and standard deviation of each DCT coefficient across all blocks and across the three YCbCr channels: luminance (Y), blue chrominance (Cb), and red chrominance (Cr). Based on these statistics, we defined two types of frequency masks: a weak mask, which retains coefficients with absolute mean less than 0.1 and absolute standard deviation less than 22; and a strong mask, which allows a broader range of coefficients with standard deviation less than 2 under the same mean threshold. Notably, the luminance channel was excluded from the masking process. This decision was based on two factors: (1) its coefficients did not meet the criteria required for masking, and (2) luminance plays critical role in human visual perception, making its full preservation essential for maintaining perceptual quality. These masks were applied uniformly to both the Gaussian samples and the ground-truth targets to improve alignment and suppress irrelevant high-frequency components.
The weak bas-b mask achieved a slightly improved FID score compared to the baseline using a naive Gaussian source distribution, as shown Table g This improvement is attributed the suppression of unnocessary high-frequency details both the source and the target, which promotes better alignment. However, the strong DCT mask led to decline in performance relative to the Gaussian baseline, suggesting
22

---

Published in Transactions on Machine Learning Research (12/2025)
Luminance Mean
Chrominance Cb Mean
Chrominance Cr Mean
14
6
6.0
0.0
0.0
-0.0
-0.0
2.7
-0.0
0.1
-0.0
4.0 0.0
-10
0.2
0.0
0.0
0.0
-0.0
10
-0.1
4.0
-0.0
0.0
0..0
10
0.8
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.3
0.0
0.0
0.0
0.3
0.0
0.0
0.0
-15
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.5
0.0
0.0
-0.0
-0.0
0.0
-0.0
0.0
-20
0.0
0.0
0.0
6.0
0.0
4..0
0.0
-0.0
-20
-0.0
-0.0
0.0
-0.0
0.00
0.0
-0.0
-0.0
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.1
0.0
0.0
0.0
0.0
0.0
0.0
-25
-0.1
0.0
0.0
0.0
0.0
0.00
0.0
-0.0
-20
0.0
-0.0
-35
0.1
Luminance Std
Chrominance Cb Std
Chrominance Cr Std
32.0 22.6 15.7 10.8
112.1
25.5
14.0
266
1.6
25.2
13.8
8.7
5.9
350
1.3
80
60
53.5
35.3
27.5
21.1
16.2
12.3
9.0
6.6
9.9
6.4
4.8
3.6
1.9
1.3
0.9
10.0
6.4
4.8
3.5
2.6
1.8
0.9
36.9
25.4
20.3
16.1
12.8
9.8
1.3
5.1
6.1
1.3
3.3
2.5
19
141
1.0
0.8
6.5
1.3
3.3
2.5
19
14
1.0
0.8
40
150
25.4
18.1
14.9
12.1
9.7
7.6
5.8
4.3
4.1
2.8
2.2
1.8
144
1.1
0.8
0.6
4.2
2.9
2.3
1.8
1.4
8.8
0.6
100
17.1
4.3
0.5
20
1.2
20
10.8
3.0
0.4
Figure 6: Mean and standard deviation DCT coefficients computed across all D D blocks in the CIFAR-10 datasot, separately for the Y (luminance), Cb (bluo-difference chrominance), and Cr (red-difference chrominance) channels. These statistics were used to construct the frequency masks by identifying coefficients with low absolute mean and variance.
Table 8: FID scores for different masking strategies. The DCT-Weak and DCT DCT-Strong methods apply frequency masking based on dataset-specific DCT coefficient stalistics, while the Gaussian baseline applies no masking.
Method
Gaussian
DeT-Weak DCT-Strong
FID4
4.30
4.20
4.39
limitations of this approach. Since the masking is applied symmetrically to both the source and the target, increasing the masking strength can degrade the fidelity of the target representation, ultimately reducing model performance.
E Explanation on why Norm Alignment Hurts at Low NFE
This issue stems from the fundamental change transport geometry when both source and target distributions are constrained similar norm hyperspheres. Under standard flow matching without norm alignment, trajectories typically follow relatively straight paths from the outer Gaussian source toward the inner data manifold. However, when norm alignment is applied, both distributions exist similar radii, fundamentally altering the transport dynamics.
As the transport operates within a batch with a finite number of samples, samples often tend to pair with targets from region with higher density, even with optimal transport pairing. This causes the learned vector field to curve slightly toward other data-concentrated regions rather than following perfectly straight paths the nearest points. When trajectories span from large to small radii (standard case), this pairing bias has minimal impact on path straightness. However, when both source and target exist similar hyperspheres
23

---

Published in Transactions on Machine Learning Research (12/2025)
Gouszian on Scurce
Galissiann Model Output Data
Model Output
aa)
(b)
Figure 7: Flow matching visualizations with Norm Alignment. (a) I-CFM with Gaussian source (b)
OT-CFM with Gaussian source
(when norms are aligned), the same pairing bias results in curved trajectories as the vector field gets "pulled' laterally toward high-density regions rather than following direct radial paths.
This curvature becomes problematic at very low NFE, because ODE solvers require more integration steps to accurately approximate curved paths compared to straight ones. With insufficient steps, the solver's linear approximations between steps fail to capture the trajectory curvature, leading accumulation of integration errors and degraded sample quality. As visualized in Fig. 7 the norm alignment fundamentally changes the transport geometry, causing trajectories to curve. This curvature is poorly approximated by ODE solvers with few steps, leading to the observed degradation in sample quality.
F Experimental Setup of High-dimensional Validation
We conduct experiments on CIFAR-10 (Kiizhvsky & Hinton 2009) and ImageNet64 (Deng et al. 2009 We adopt improved UNet (Ronneberger et al. 2015) architecture with attention, following the ADM framework (Dhariwal & Nichol 2021). Both models use basc channel width of 128, four attention heads with 64 channels each, Group Normalization, GELU activations, dropout rate of 0.1, and convolutional resampling. For CIFAR-10 (32 x 32), the model uses residual blocks per resolution and applies attention at 16 16. For ImageNet64 (64 x 64), we use residual blocks with attention at 32 x 32 and 16 x resolutions.
Training is performed using the Adam optimizer with learning rate of x 10 and a linear warmup schedule. For CIFAR-10, train for 200K steps with batch size of 512 and 5K warmup steps. For ImageNet64, use 350K steps, a batch size of 384, and 20K warmup steps. All experiments are run on single NVIDIA RTX A6000 (48GB) GPU.
G Source Pruning Settings
For source pruning, we empirically set the pruning thresholds to 0.01, T, 0.048 for CIFAR-10 and 0.005, 0.026 for ImageNet64.
To provide further insight into the effect of these hyperparametors, we conducted an ablation study on CIFAR-10 by varying the Uhreshold 7,. Table 9 reports the Fréchet Inception Distance (FID) scores, total generation time for 512 images, and the corresponding rejection rates. The results illustrate trade-off: as T increases, total computation time decrcases, but this is accompanied by a slight degradation in sample quality (higher FID). Nevertheless, our pruned sampling approach consistently outperforms the standard Gaussian sampling baseline (r, =1.0) in FID, even at very low rejection rates with additional computation.
24

---

Published in Transactions on Machine Learning Research (12/2025)
Table 9: Ablation study on the pruning threshold t, for CIFAR-10. FID scores are reported for both 1CFM and our OTCFM. The total time corresponds to generating 512 images.
FID
Rejection Single Image (ms)
Tt
(I-CFM OT-CFM)
Total Time (s)
Rate (%)
0.05
4.00
4.10
19.96
38.98
99.94
0.055
4.05
4.10
18.11
35.37
94.90
0.06
4.13
4.21
18.09
35.32
70.08
0.07
4.22
4.28
18.09
35.32
11.88
0.08
4.24
4.33
18.09
35.32
0.97
1.0 (Ganssian)
4.36
4.40
18.09
35.32
0.00
H Norm Scaling of 2D Simulations
As described in Section 3.2 scale the approximated source so that its average norm matches the average radius of the x-Sphere. This normalization ensures that all source types start from a comparable average norm, allowing us to focus on visualizing the transport path patterns relative to the x-Sphere. While this norm scaling may reduce the apparent convergence of the source to the data, it provides a clearer and more consistent visualization of the transport dynamics across different source settings. Without this normalization, the dynamics become difficult interpret, demonstrated in Fig.
Generated
Generated
Generated
Data
Data
Data
(a)
(b)
(c)
Figure 8: Visualization of flow matching with density-approximatee source without norm scaling. (a) OT-CFM with Approximated Source (iter 200), (b) OT-CFM with Approximated Source (iter 6000), (c) OT-CFM with Approximated Source (iter 10000).
Limitations
While our study provides a conceptual understanding of the learning dynamics of flow matching and valuable insights into the design of source distributions for flow matching, several limitations remain.
First, our conclusions primarily derive from experiments on image datasets such as CIFAR-10 and ImageNet64. These datasets may not fully capture the challenges of other modalities like text, audio, or molecular data, where distribution geometries and semantics differ significantly. This also applies to latent spaces, such
those from image tokenizers (Rombach et al.) 2022 Esser al. 2024 Podell et al. 2023 Yao et al. 2025 Chen et al.] 2025 Lee et al.) 2025 Chen et al. 2024), where the structure of embeddings can vary widely. Recent studies investigating the geometric properties of latent spaces such as analyzing intrinsic manifold curvature (Arvanitidis et al. 2017 Choi et al. 2021) or learning structured, isometric representations (Preechakul et al. 2022 Hahm al.. 2024) (224)-igghighh the complexity of those domains. The generalizability of our findings these domains remains an open question. Second, our analysis of several source distributions offers practical insights but falls short of establishing comprehensive theoretical
25

---

Published in Transactions on Machine Learning Research (12/2025)
foundation for optimal source distribution design in high-dimensional spaces. Further work is needed to develop rigorous guarantees and theoretical bounds that can guide more principled approaches to source distribution optimization across diverse application domains. Third, while source pruming and norm alignment improve performance, they introduce additional byperparameters- lactors that require careful luning and incur additional computational overhead due to rejection sampling. Morcover, their effectiveness sensitive to the number of function evaluations (NFEs). Notably, norm aligument may degrade performance in very low-NFE regimes, presumably due increased curvabure in the generative trajectory. Finally, our study focuses on unconditional generation. The applicability of our findings to conditional generation tasks, such as class-conditional or text-to-image generation, remains unexplored. Various conditioning mechanisms can interact with the source distribution design to change the effectiveness of the proposed stralegy, which requires future research.
as pruning thresholds and scaling
Despite these limitations, our work establishes important empirical guidelines for source distribution design and provides foundation for future rescarch in this area. We believe that addressing these limitations through expanded empirical evaluation, theoretical analysis, and computational optimization will be valuable directions for advancing flow matching methods.
J Qualitative examples
Original
Generated
Original
Generated
Figure 9: Left-most images (red frame) of each cluster are target samples the five images on the right are generated from up ~ vMF(u((1),,) x = 1500) by the generator. Although direct pairing to concentrated areas raised concerns about potential overfitting to the target data, we observe that the generated images remain diverse and visually distinct while being similar.
26

---

Published in Transactions on Machine Learning Research (12/2025)
(a) DCT Weak: source
(b) DCT Weak: generated
(c) DCT Strong: source
(d) DCT Strong: generated
(e) GMM-1: Source
(f) GMM-1: generated
GMM-2: source
(h) GMM-2: generated
(1) GMM-10: source
(j) GMM-10: generated
(k) FFJORD: source
()) FFJORD: generated
Figure 10: Source-vs. gen-eeerated 2-D samples for discrete mixtures (Part I). Each adjacent pair visualises the original training distribution (left) and the output of the generator (right) for DCT-WEAK DCT-STRONG, three GMM settings, and FFJORD.
27

---

Published in Transactions on Machine Learning Research (12/2025)
(a) source
(b) K 10: generated
K 50: source
(d) k 50: generated
100: source
(f) 100: generated
(g) 300: source
(h) 300: generated
(1) 900: source
(j) k 900: generated
(k) 1500: source
(1) 1500: generated
Figure 11: compare the empirical data (left) with model outputs (right) increasing concentration parameters from 10 to 1500.
2-D samples for von Mises Fisher distributions (Part II). Pairs
28


Reviews:
[{"review_id": "82wNd86Va2", "reviewer": "Reviewer_kdcC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a new 2D simulation method for analyzing the dynamics of flow matching models during training. Using the insights from the small-scale setting, they make a number of observations such as density approximation degrading performance, directional alignment suffering from path entanglement, gaussians providing more robust coverage, and norm misalignment causing poor learning dynamics. The work proses a framework to reconcile some of the observed issues such as aligning norms while directionally pruning samples. The method empirically supports their findings through experiments. They find improved generation quality with more efficient sampling.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the experiments back the claims. I encourage the authors to perform experiments on larger-scale diffusion models as the largest dataset they benchmark on is ImageNet64. As the method doesn't require retraining, it would not be too difficult to evaluate the method on state-of-the-art models. Showing that this method works for cutting edge models would significantly increase the impact of the work.", "audience_yn": "Yes", "audience_explanation": "The insights and analysis of the 2D learning dynamics are interesting and likely relevant to the members of the community. I am unsure whether insights from the novel 2D simulation will generally translate to the high-dimensional embedding space that typical diffusion models operate in, but it is worth investigating. \n\nThe proposed framework could be very useful if the efficiency and performance gains work for large-scale models.", "requested_changes": "I encourage the authors to perform more extensive experiments on large-scale diffusion models as I discussed above in the claims and evidence section. Though the method may work on small-scale models trained on cifar and ImageNet64, it would be nice to show that it translate to standard open-source models such as Stable Diffusion. It is not critical to acceptance, but would greatly increase the impact and usefulness of the method.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JqCPBMWqL3", "reviewer": "Reviewer_RrVx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper discuss the source distribution for flow matching and proposes a practical frameworks that combines norm0aligned training with directionally-pruned sampling. The proposed approach facilitates stable flow learning while eliminating data sparse-region initialization during inference. Experiment on CIFAR10 demonstrates the advantages of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors provide both theoretical and practical support of their findings.", "audience_yn": "Yes", "audience_explanation": "The discussion on flow matching is important.", "requested_changes": "1. Add experiments on larger dataset.\n2. Add experiments on difference tasks, e.g., image inpainting, image restoration.", "broader_impact_concerns": "n/a", "additional_comments": "n/a", "extra_scores": {}, "extra_text": {}}, {"review_id": "TVgwscdFUj", "reviewer": "Reviewer_7yfb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper contributes to the expanding literature of shaping (non-gaussian) source distributions for flow-matching in a data-driven manner.     \n\nSpecifically, the authors explore a number of cases, both synthetic and realistic which reflect complex high-dimensional geometries.\n\nThe synthetic dataset is constructed is two-dimensional, but designed to mimic high-dimensional geometry through a spherical coordinate decomposition - with 3 clusters of data of unequal size used to mimic complex data in typical high-dimensional settings.\n\nBased on various experiments using these datasets, the authors explore various strategies on exploiting data-geometry to identify better source distributions.   They conclude the following: \n1. density-approximating sources (e.g., generating samples from a GMM, CNF based on training data)  hurt due to missed low-density modes (\"mode discrepancy\"), increasingly so the stronger/tighter the fit of the density approximation is, due to missed,  low mass modes.\n2.  tightly aligning the directions of source data with training data can induce “path entanglement” and instability unless pairing is exact; \n\nBased on these experiments the authors propose two approaches.\n1. Use plain Gaussian for training, which works omnidirectional coverage.\n2. Augment this with  Norm-Alignment:  which scales target norms during training to match the source (and rescales back at the end).\n3. Pruning at inference: i.e. using rejection sampling to remove Gaussian samples in data-sparse  directions.  This improves FID and sampling robustness without retraining. Empirical support spans the synthetic data plus CIFAR-10 and ImageNet64.\n\nStrengths: \nThe paper is largely empirical and exploratory, but does provide useful and pragmatic suggestions around source distribution construction, making it a valuable contribution.    The datasets considered are not expansive - one synthetic dataset and 2 image datasets, but the synthetic dataset and the way its constructed do elucidate the various mechanisms involved.  \n\nThe mechanism of entanglement is interesting identification of a failure more for source distribution selection, for which the authors derive a informal analysis of the lipschitz constant of the flow-field and its dependence on cap-geometry constants and concentration.\n\nWeaknesses: \n1. While the paper does demonstrate efficacy of these approaches on real data, I do feel that the paper could better address other challenging types of data geometries.  For highly multimodal distributions of real data, one cannot magically expect to find an origin with respect to which the data spreads out in angular modes.   One would probably also expect significant radial multi-modality,  where there are multiple clusters of data with the same angle but different radii.  An extreme case would be concentric rings of data-points, radiating out from the origin.     It is unclear whether these approaches would be effective in that setting, or if other mitigations are necessary.\n2. It would have been nice to see some other datasets beyond the CIFAR, Imagenet ones,  maybe a latent model or a conditional model, which are the most challenging settings for FM - appreciate that its addressed in the limitations", "claims_and_evidence_yn": "Yes", "claims_explanation": "Mostly yes.\n\nThe main claims are:\n\n1. Gaussian->Pruned improves FID without retraining.\n2. Norm Alignment helps (mostly at higher NFE) and can hurt at very low NFE.\n3. Density approximation can hurt. DCT (mild) slightly helps; stronger GMM/CNF approximations degrade markedly.\n4. Directional alignment: tight vMF + minibatch OT underperforms; oracle/global pairing shines - the authors identify path entanglement as the failre mode\n5. 2D $\\chi$-sphere simulation backs the story with experiments.  The appendix reports mean±std over 10 runs across multiple metrics, and the construction is explicitly derived.", "audience_yn": "Yes", "audience_explanation": "This paper contributes to the wider growing area of flow-matching, and would be of interest to those developing flow-matching models for their own applications, or interesting in developing novel approaches to flow-matching.", "requested_changes": "- Figure 2 is quite hard to interpret - I would expect that as iter increases, that the source points get closer to the data, but I don't see this.\n- Pruned Sampling: You define $\\tau$, $\\tau_r$  and the PCA test, but don’t report acceptance rates or typical resamples per kept sample. It would be useful to include both and a note on compute overhead. e.g. What % of candidates pass for CIFAR-10/ImageNet64 at your reported parameter values.  Maybe adding a small appendix plot: FID vs retained fraction (sweeping over $\\tau$, $\\tau_r$​), and one varying PCA dimensionality $k$ if you use a truncated basis. \n-\n- More practically, how do you choose these hyperparameters $\\tau$, $\\tau_r$, $k$, etc.\n\n* The Tables with the image experiments list single numbers; it would be nice if you could report report mean $\\pm$ std over multiple seeds, as is done for the 2D sim in Figure 7.  This would dispel the objection that some of the FID changes could be run noise.\n\n* Are you able to explain why alignment hurts at very low NFE?\n\n* In ImageNet64 you say \"Euler integration\"—is CIFAR-10 also Euler for the OT-CFM runs?\n\n* Is the radial multimodality example described above an issue for your current approach, and if so, how can it be addressed?\n\n* DCT: the weak/strong outcomes seem buried in the appendix - might be worth bringing them out.\n\n\nMinor issues:\n\nD1. Practive -> Practice\nD1. appled -> applied.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)