Title: Sample and Map from a Single Convex Potential: Generation using Conjugate Moment Measures
Venue: NeurIPS 2025
Accepted: yes
Keywords: optimal transport

generative modelling

input convex neural networks
Abstract: The canonical approach in generative modeling is to split model fitting into two blocks: define first how to sample noise (e.g. Gaussian) and choose next what to do with it (e.g. using a single map or flows). We explore in this work an alternative route that ties sampling and mapping. We find inspiration in moment measures, a result that states that for any measure $\rho$, there exists a unique convex potential $u$ such that $\rho=\nabla u \sharp e^{-u}$. While this does seem to tie effectively sampling (from log-concave distribution $e^{-u}$) and action (pushing particles through $\nabla u$), we observe on simple examples (e.g., Gaussians or 1D distributions) that this choice is ill-suited for practical tasks. We study an alternative factorization, where $\rho$ is factorized as $\nabla w^* \sharp e^{-w}$, where $w^\ast$ is the convex conjugate of a convex potential $w$. We call this approach conjugate moment measures, and show far more intuitive results on these examples. Because $\nabla w^*$ is the Monge map between the log-concave distribution $e^{-w}$ and $\rho$, we rely on optimal transport solvers to propose an algorithm to recover $w$ from samples of $\rho$, and parameterize $w$ as an input-convex neural network. We also address the common sampling scenario in which the density of $\rho$ is known only up to a normalizing constant, and propose an algorithm to learn $w$ in this setting.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method for generative modeling based on optimal transport and moment measures. The method links the sampling phase and the mapping phase, which are typically treated as completely separate in most generative modeling approaches. First, the authors recall a result on moment measures stating that for any measure ρρ (satisfying certain conditions), there exists a unique convex function $ u $ such that  $ \\rho = \\nabla u $ # $  e^{−u} $. They then observe, using a simple Gaussian example, that this factorization is not well suited for generative modeling. The authors subsequently propose an alternative factorization: there exists a unique convex function $ w $ such that  $ \\rho = \\nabla w^* $ # $  e^{−w} $ where $ w^* $ is the convex conjugate of $ w $. They propose learning the convex potential $ w$  using Input-Convex Neural Networks (ICNNs), leveraging recent advances in neural optimal transport solvers, and then sampling from $ e^{-w} $ using a Langevin dynamics scheme. Finally, the authors demonstrate the effectiveness of their method for generative modeling on low-dimensional toy examples and small synthetic image datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThis is a good theoretical paper that is well written. The maths, as far as I can tell, seem to be correct, and the method appears to be original and novel. Although the experimental results suggest that the method cannot yet compete with state-of-the-art generative modeling approaches—even on relatively simple tasks such as 2D toy examples or MNIST generation—I still consider it a promising new direction with solid theoretical foundations and a strong connection to optimal transport.\n\nThat being said, I believe the experimental section could be slightly improved. In my opinion, it lacks commentary on failure cases, such as the checkerboard patterns in 2D, for instance. Additionally, the ICNN architecture used for the MNIST experiment seems overly simple compared to standard models typically applied to this dataset. It would have been helpful to test a more expressive architecture—at least a convolutional neural network (CNN)—to better assess the potential of the method for generative modeling.\n\nLastly, I think the paper would benefit from a brief discussion that situates the contribution within the broader field of generative modeling. In particular, it would be useful to outline the potential advantages of this method over existing alternatives.\n\nQUESTIONS:\n- 1/ If $ \\rho = \\nabla u$  #  $ \\mathfrak{B}_u = \\nabla w^* $ #  $ \\mathfrak{B}_w $, do we have $ \\mathfrak{B}_u^*= \\mathfrak{B}_w $? \n\n- 2/ What could be the impact of the fact that your results only hold for absolutely continuous measures, given the manifold hypothesis, which suggests that real data distributions are highly degenerate?\n\n- 3/ Is $ \\nabla w^* $ still an OT map? \n\n- 4/ From the experiments, it appears that the results are often noisy. Do you have an intuition to explain this?\n\nLIMITATIONS:\nI think the limitations of the method could be a bit more discussed, relatively to the potential discussion that better situates the contribution already mentioned in section 'Strengths and Weaknesses'.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces conjugate moment measures for sampling. Drawing inspiration from moment measures, which enable the expression of a measure $\\rho \\in \\mathcal P(\\mathbb{R}^d)$ as $\\rho = \\nabla u_\\sharp e^{-u}$ for some convex $u: \\mathbb{R}^d \\to \\mathbb{R}$, the paper shows that any measure $\\rho \\in \\mathcal P(\\mathbb{R}^d)$ that is absolutely continuous with compact, convex support can also be written $\\rho = \\nabla w^{\\ast}_{\\sharp} e^{-w}$, where $w: \\mathbb{R}^d \\to \\mathbb{R}$ is a convex potential and $w^{\\ast}$ is the convex conjugate of $w$. The paper additionally shows the existence of such a conjugate moment decomposition for Gaussian measures with non-degenerate covariance matrices and argues, based on examples for two-dimensional Gaussian and one-dimensional Gaussian mixture distributions, that conjugate moment measures are better suited for sampling than moment measures.\n\nThe paper then proposes computational strategies for sampling from $\\rho$ via its conjugate moment measure decomposition under three different information scenarios: (1) when $w$ is known, (2) when existing samples from $\\rho$ are known, and (3) when the unnormalized density of $\\rho$ is known. Estimation of $w$ from samples involves a fixed-point algorithm wherein a Brenier potential is computed on each step, either in closed form for 1D distributions or by optimization of an ICNN using neural optimal transport. Estimation of $w$ from a density entails training an ICNN on a collocation loss corresponding to the Monge-Ampere equation (a-la PINNs). In either case, once $w$ is in hand sampling is performed by first using Langevin Monte Carlo (LMC) to sample from $e^{-w}$ and then using $w^*$, identified via gradient-based optimization, to push-forward these samples to $\\rho$. The numerical examples in the paper demonstrate proof-of-concept.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths \n* This paper is clearly written and easy-to-follow. \n\n* The result showing existence of conjugate moment measures is interesting and new and follows from a nice fixed-point argument. \n\n* The results and figures concerning the moment potentials and conjugate moment potentials of Gaussians are helpful for building intuition.\n\n* The experimental details are clearly detailed in the appendix and would facilitate reproduction of these results by others.\n\n## Weaknesses\n* The argument that moment potentials are unsuitable for practical sampling tasks and that conjugate moment potentials offer a better alternative is primarily based on a relatively narrow set of examples -- Gaussians and Gaussian mixtures in one and two dimensions.  I am not convinced that the behavior in these examples generalizes to non-Gaussian (mixture) and higher-dimensional distributions based on the arguments given in the paper. \n\n* Moreover, moment measures/potentials are not typically used for practical sampling tasks, so from that standpoint the idea that we need an alternative to the moment potential factorization isn't credible. \n\n* The computational effort required to sample via a conjugate moment measure is high compared to other sampling methods, and the extra computation has not been adequately justified.  \n\n* The numerical results include no comparisons to widely used or state-of-the art sampling methods. The only comparisons are to ICNNs trained to push forward a Gaussian to $\\rho$ directly, whereas state-of-the-art neural network transport approaches usually invoke some kind of dynamics, rather than being \"one-shot\".\n\nQUESTIONS:\n* Regarding the increased computational cost relative to other methods: if we have the density of $\\rho$, in order to sample from $\\rho$ using a conjugate moment potential we have to (1) train a PINN (which specifically must be an ICNN) to obtain $w$, (2) run LMC to get samples of $e^{-w}$, and then (3) compute $\\nabla w^*(x)$ for every sample $x$ generated by LMC. The pipeline for generative modeling with conjugate moment measures is similar. There are many sampling methods that enable the user to generate new samples directly after the training of a neural network (neural ODEs, flow-matching, using PINNs to solve continuity equations, etc.), without the post-processing steps involved in the proposed method. In the density driven-setting, one could run LMC on $\\rho$ directly. How does one rationalize the extra costs associated with using conjugate moment measures to sample? Have you compared the quality of samples generated by the conjugate moment measure with those generated by a trained neural network (for dynamic transport) directly or by LMC? \n\n* Theorem 1 guarantees existence of a conjugate moment factorization for absolutely continuous measures $\\rho$ with compact, convex support. Many of the distributions in the numerical examples do not satisfy these assumptions. Is this a problem? How do you know, for example, that the fixed-point iteration in CMFGen will converge? Have you tried any examples where it didn't converge? \n\n* It appears that noise is added to the generative modeling training images to guarantee existence of an OT map. Computationally, does the pipeline break down if noise is not added?\n\n* The face image inpainting example seems relatively easy because the mask covers one vertical half of the the image and faces are symmetric about the vertical axis. I could inpaint the images by simply reflecting the unmasked half across the vertical axis. What happens if you mask the top or bottom half of the image instead of a left or right half? Is the quality of the in-painting as good, or does it get degraded?\n\nLIMITATIONS:\nIt appears that in the generative modeling setting noise must be added to images before training to ensure the existence of an OT map. This requirement seems like a major limitation for generative modeling, as it would preclude the generation of noiseless images, but it is not discussed outside of one line in the appendix. Do the authors see a way that this step could be avoided? Or do they see a way that their conjugate moment factorization could play an intermediate role in generative modeling pipelines which ultimately produce noiseless images? Further discusson of these limitations and possibilities in the conclusion would help contextualize the results of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe standard approach in generative modeling separates noise sampling (e.g., Gaussian) from mapping (e.g., flows). The authors propose an alternative that links sampling and mapping, inspired by moment measures, which connect log-concave distributions and convex potentials. However, this method is impractical for common tasks. Instead,  ρ is factorized as ∇w* ♯ e^−w, with w* the convex conjugate of w.  This is called the conjugate moment measure approach. Using optimal transport and input-convex neural networks, w is recovered from samples, even when ρ’s density is only known up to a constant. This yields more intuitive results in practice.\n\nSTRENGTHS AND WEAKNESSES:\nStengthes: This paper presents a novel approach to generative modeling by proposing an alternative to the traditional two-step process of noise sampling and mapping. Drawing inspiration from the theory of moment measures, the authors explore a framework that directly couples the sampling and mapping steps.  Specifically,  the paper introduces the idea of \"conjugate moment measures,\" where the target distribution is represented via the Monge map associated with the convex conjugate of a potential function. The authors use optimal transport solvers and parameterize the potential using input-convex neural networks.  \n\nWeakness: The main weakness of this work is that the computational cost is not dsicussed in detail. For sampling \nsampling from ρ using its Conjugate Moment Factorization, it requires concave maximization problem:∇w∗(y) = arg supx ⟨x, y⟩ − w(x) in each iteration. It is not how expensive this is computaitonally, especially in high-dimensional settings. Second, there is no discussion on the advantages/disadvantages of the proposed method relative to the existing generative methods, such as diffuion and continuous normalizing flows. Finally, the experimental results are not convicining. While the results show some promise of the method, more experiments with higher-dimensional image data will  significantly strengthen the paper.\n\nQUESTIONS:\nI have the following questions:\n\n(1) Is the conjugate moment measure factorization given in Theorem 1 unique? \n\n(2) What is the  computational complexity the proposed algorithm?\n\n(3) How scalable the algorithm is?\n\n(4)  I wonder if the authors can comment on the statistical errors of the generated distribution \n      based on the proposed method.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to a novel approach to generative modeling, using a variant of optimal transport. Rather than relying on a heuristically chosen initial distribution, which is possibly the biggest hyperparameter in generative modelling, they propose a method which constructs the base distribution along with the pushforward map to the data distribution. They build upon known theoretical tools like the brenier potential and moment measure to eliminate the limitations of typical optimal transport based methods which are prone to create very ill-conditioned maps. They demonstrate the results on some synthetic low dimensional datasets as well as MNIST and Cartoon dataset.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper developes a critical idea to eliminate one of the more important hyper parameters in generative modelling, the initial distribution. The known approaches to picking this distribution are mainly heuristic, and this proposed alternative provides a good approach to this problem. The main result of the paper is the existence of conjugate moment potential, which is proved via a fixed point theorem applied to the Brenier potential map. I found the proof to be fairly involved.\n\nPaper presents a fairly convincing argument about why conjugate moment potentials are preferred over moment potentials. The argument is limited to a Gaussian distribution, but suffice to convince the ill-conditioned nature of moment potentials.\n\nThe paper has limited experimental coverage with focus on low-dimensional synthetic datasets along with some experiments on high dimensional datasets like MNIST and Cartoon. It would be interesting to see how the experiments scale, but I think current experiments suffice for the purpose of this paper whose main goal is to explore a novel modelling idea.\n\nQUESTIONS:\nHigh level questions:\n1. Can you please provide a complete and formal statement of Breiner's thorem that is being used either in the main paper or in Appendix?\n2. From what I understand, although the theory holds for convex potentials, none of the methods are actually using convex potentials. Is that correct? If not, how are you ensuring that $w_\\theta$ are convex? If yes, how is convex conjugate / legendre transform interpreted for a non-convex function?\n3. Would it be possible to compare performance with other energy-based modelling methods (like say score matching) in a synthetic setting for example over an exponential family?\n4. Line 214: How is $w_\\theta$ initialized to be roughly $\\lVert \\cdot \\rVert^2$? It does not feel necessary to have this initialization for the proposed method, can you explain choice of this initialization?\n\nQuestions/comments about the proofs:\n1. Line 404: $w$ is convex only if $\\Omega$ is convex. Is this assumed?\n2. Line 405: How is $\\nabla w$ defined? $w$ seems to be discontinuous on boundary of $\\Omega$.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops a new, mathematically-motivated method for generative modeling. The approach is based on a so-called moment measure factorization of Cordero-Erausquin and Klartag [2015] that provides a factorization of any measure $\\rho$ as $\\rho = \\nabla u \\sharp e^{-u}$ where $u$ is a convex potential function. The authors observe that this factorization can be ill-conditioned, and instead propose to leverage a novel, related factorization $\\rho = \\nabla \\omega^* \\sharp e^{-\\omega}$ where $\\omega$ is convex and $\\omega^*$ denotes its convex conjugate. The approach is demonstrated on several simple numerical examples, and some theory is proven demonstrating the validity of the approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper is very well-written and is a pleasure to read.\n- The idea is novel and interesting, and leverages deep mathematical results that have not yet been applied in the generative modeling space, implying a significant level of originality.\n- The method is mathematically very well-motivated and grounded, which is refreshing, particularly in a space where engineering advancements can overpower theoretical ones.\n\n\n**Weaknesses**\n- The notation can at times be difficult to read -- for example $*$ is used both to denote the optimizer and the convex conjugate, and there are several usages of related letters such as $\\mathcal{B}$ and $\\mathfrak{B}$ for distinct concepts.\n- The method seems to be very expensive numerically, particularly given its relation to optimal transport. Moreover, the sampling variant requires the ability to sample from the target, which feels circular.\n- Most significantly, despite its elegance, the method doesn't work very well: even on the two-dimensional checker example, for which we can now obtain very good results using flow matching or diffusion models in a matter of minutes, the results are pretty poor. The results on higher-dimensional datasets like images are unfortunately quite bad, even for simple cases such as MNIST.\n- Very minor comment, but the authors miss a few of the major initial references on flow matching, namely https://arxiv.org/abs/2209.15571, https://arxiv.org/abs/2303.08797, and https://arxiv.org/abs/2209.03003\n\nQUESTIONS:\n1. In the introduction, the authors comment that $\\omega^*$ is the Brenier potential linking $e^{-\\omega}$ to $\\rho$. Can the authors describe, then, why the method is different from optimal transport? Is it simply because we also learn the source distribution? We know that the optimal transport map can be hard to learn, and so many generative modeling approaches are intentionally based on sub-optimal transport to simplify the estimation problem. Can the authors explain clearly why it is a \"good idea\" to try to estimate the optimal transport map if the goal is \"only\" generative modeling?\n\n2. Can the authors explain the computational efficiency of their approach? The iterative optimal transport computation in Sec. 4.2 seems very expensive.\n\n3. Can the authors comment on the feasibility of their sampling method from Sec. 4.3? The sampling method requires Langevin samples from the target measure $\\rho$ to learn. This seems to be a chicken and an egg problem, because the method's goal is to generate samples from $\\rho$. From this perspective, I'm having trouble seeing if the method will really work in practice. Is there a way to make it iterative and sample from the current estimate of $\\rho$, using the fact that $\\omega$ is convex? Will this suffer from mode collapse?\n\n4. Can the authors get their method to work better in the numerical examples? I'm a firm believer that we shouldn't require SoTA image results in every paper on generative modeling, but I do think if the method doesn't really work on the 2d checker, it indicates a problem. For publication, I think the authors need to have compelling results on some simple examples. They should also try to quantify their results in some way, such as FID on images and metrics such as KL or Sinkhorn in low-d. The authors should also tone down the claims in the text that the method works well, which are sort of contradicted by their figures.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-08"}