Title: Deep Legendre Transform
Venue: NeurIPS 2025
Accepted: yes
Keywords: Convex Conjugates

Legendre Transformation

Deep Learning

Input Convex Neural Networks
Abstract: We introduce a novel deep learning algorithm for computing convex conjugates of differentiable convex functions, a fundamental operation in convex analysis with various applications in different fields such as optimization, control theory, physics and economics. While traditional numerical methods suffer from the curse of dimensionality and become computationally intractable in high dimensions, more recent neural network-based approaches scale better, but have mostly been studied with the aim of solving optimal transport problems and require the solution of complicated optimization or max-min problems. Using an implicit Fenchel formulation of convex conjugation, our approach facilitates an efficient gradient-based framework for the minimization of approximation errors and, as a byproduct, also provides a posteriori estimates of the approximation accuracy. Numerical experiments demonstrate our method's ability to deliver accurate results across different high-dimensional examples. Moreover, by employing symbolic regression with Kolmogorov–Arnold networks, it is able to obtain the exact convex conjugates of specific convex functions.

Deep Legendre Transform

Abstract

We introduce a novel deep learning algorithm for computing convex conjugates of differentiable convex functions, a fundamental operation in convex analysis with various applications in different fields such as optimization, control theory, physics and economics. While traditional numerical methods suffer from the curse of dimensionality and become computationally intractable in high dimensions, more recent neural network-based approaches scale better, but have mostly been studied with the aim of solving optimal transport problems and require the solution of complicated optimization or max-min problems. Using an implicit Fenchel formulation of convex conjugation, our approach facilitates an efficient gradient-based framework for the minimization of approximation errors and, as a byproduct, also provides a posteriori estimates of the approximation accuracy. Numerical experiments demonstrate our method's ability to deliver accurate results across different high-dimensional examples. Moreover, by employing symbolic regression with Kolmogorov-Arnold networks, it is able to obtain the exact convex conjugates of specific convex functions. Code is available at https://github.com/lexmar07/Deep-Legendre-Transform

1 Introduction

In this paper, we introduce DLT, a deep learning method for computing convex conjugates, which play an important role in different fields such as convex analysis, optimization, control theory, physics, and economics.

The modern formulation of convex conjugation, also called Legendre-Fenchel transformation, was introduced by Fenchel [1949] 1 and defines the convex conjugate of a function f : C → R on a nonempty subset C ⊆ R d by

This definition does not need any special assumptions on the function f or the set C and makes sense for all y ∈ R d . Moreover, it directly follows from (1.1) that the conjugate function f ∗ : R d → ( -∞ , ∞ ] is lower semicontinuous and convex.

In the special case, where f : C → R is a differentiable convex function on an open convex set C ⊆ R d , one has for all y ∈ D = ∇ f ( C ) . The right side of (1.2), called Legendre transform, was used by Legendre [1787] when studying the minimal surface problem. Note that for it to be well-defined, the gradient map ∇ f : C → D does not have to be invertible. In fact, it can easily be shown that if f is differentiable and convex, the right side of (1.1) is maximized by any x ∈ C satisfying ∇ f ( x ) = y . So the preimage ( ∇ f ) -1 ( y ) in (1.2) can be substituted by any point x ∈ ( ∇ f ) -1 ( y ) .

Since f ∗ is not always available in closed form, various numerical methods have been developed to approximate it numerically. Most of the classical approaches use the modern definition (1.1) and approximate f ∗ ( y ) for y in a finite subset Y ⊆ R d by the discrete Legendre-Fenchel transform

based on a finite subset X ⊆ C . Of course, for (1.3) to provide a satisfactory approximation to f ∗ , the subsets X and Y need to be sufficiently dense. Typically, X and Y are chosen as grids X = X 1 × . . . ×X d and Y = Y 1 × . . . ×Y d of similar size. For instance, for regular grids of the form X = Y = I d N = { 1 N , 2 N , . . . , 1 } d , conditions under which the discrete Legendre-Fenchel transform (1.3) converges to f ∗ have been given by Corrias [1996]. In this case, a naive computation of (1.3) for all y ∈ Y , has complexity O ( N 2 d ) . For the case where N is a power of 2 , Brenier [1989] and Noullez and Vergassola [1994] have derived fast algorithms to compute (1.3), similar in spirit to the fast Fourier transform of Cooley and Tukey [1965], with complexity O (( N log N ) d ) . By first passing to convex hulls, Lucet [1997] has been able to design an algorithm with complexity O ( |X| + |Y| ) , which for X = Y = I d N means O ( N d ) . But since this is still exponential in d , it essentially limits the method to dimensions d = 1 , 2 , 3 and 4 .

A different approach, going back to a tweet by Peyré [2020], uses the fact that if C ⊆ R d is bounded, f ∗ can be approximated 2 with the softmax convex conjugate

for a small ε > 0 , which can be expressed via Gaussian convolution as

where

Approximating the convolution Q ε ( f ) ∗ G ε with a sum over a finite set X ⊆ C yields a GPU-friendly algorithm with complexity O ( |X| log |X| ) ; see Blondel and Roulet [2024] for details. But since it is grid-based, it still only gives accurate results in low-dimensional settings.

Lucet et al. [2009] have explored the computation of convex conjugates of bivariate piecewise linear quadratic convex functions, and Haque and Lucet [2018] have proposed an algorithm for such functions with linear worst-case complexity. But the scalability to higher dimensions remains a challenge.

Recently, numerical methods for convex conjugation have also been studied in the machine learning literature as a tool to solve the Kantorovich dual of optimal transport problems and develop efficient Wasserstein GANs. While their performance has mostly been tested to see how well they help to solve transport problems, promising results have been obtained by training neural networks to predict an optimal x in the Legendre-Fenchel transform (1.1). E.g. Dam et al. [2019] and Taghvaei and Jalali [2019] predict x from y with a neural network h ϑ ( y ) , where h ϑ is maximizing the right side of (1.1). Makkuva et al. [2020] do the same using predictors of the form h ϑ = ∇ ℓ ϑ for an ICNN 3

ℓ ϑ . Korotin et al. [2021a,b] and Amos [2023] have proposed to predict an optimal x in (1.1) with a neural network h ϑ by minimizing the first order condition

which, in effect, amounts to learning a generalized inverse of the gradient map ∇ f : C → D ; see Section 4 below.

In this paper, we employ the following implicit Legendre formulation of convex conjugation:

for differentiable convex functions f : C → R defined on an open convex set C ⊆ R d . This allows us to approximate f ∗ : D = ∇ f ( C ) → R by training a machine learning model g θ : D → R parametrized by a parameter θ from a set Θ such that

DLT implements this by solving

for a training set X train ⊆ C . This combines the following advantages:

(i) On the training points x ∈ X train , thanks to (1.4), the algorithm's target values ⟨ x, ∇ f ( x ) ⟩-f ( x ) exactly agree with the true values f ∗ ( ∇ f ( x )) .

(ii) If g θ is chosen as a neural network, the approach readily scales to high-dimensional setups, and if combined with an ICNN as approximator, it is guaranteed to output a convex approximation to the true Legendre transform, which is known to be convex.

(iii) Specifying g θ as a KAN 4 makes it possible to obtain exact solutions for certain convex functions.

(iv) The implicit Fenchel formulation (1.4) can also be used to derive a posteriori error estimates.

(i) is a theoretical advantage compared to existing approaches and, as illustrated in Section 5 below, yields good numerical results, in fact, limited only by the used NN architecture/computing power available. We demonstrate that DLT achieves state-of-the-art performance by comparing it to the direct learning of the convex conjugate (i.e., using f ∗ as a target function in cases where it is available in closed form).

(ii) ICNNs have been introduced by Amos et al. [2017], and it has been shown by Chen et al. [2019] that they can approximate any convex function arbitrarily well on compacts. With neural networks as approximators, the minimization problem (1.5) can numerically be solved in high dimensions with a stochastic gradient descent algorithm. The choice of ICNNs ensures that the approximation is convex. This regularizes the learning problem and helps against overfitting. Moreover, it is helpful in various downstream tasks, such as optimization, control, or optimal transport, where convex conjugation can be utilized as a powerful tool to find optimal solutions. Alternatively, we use ResNet (He et al. [2016]), which in many multidimensional cases provides a better approximation than ICNN but does not guarantee convexity.

(iii) KANs have recently been introduced by Liu et al. [2025]. They are particularly powerful for approximation tasks requiring high precision. But in addition, they can also be used for symbolic regression to recover functions that have a closed form expressions in terms of special functions such as polynomials, exp , log , sin or cos . In Section 5 below, we show that KANs can compute Legendre transforms of certain convex functions exactly.

(iv) As a byproduct, our approach yields a posteriori L 2 -approximation guarantees. Indeed, by testing a candidate approximating function g : C → R against the implicit Legendre transform (1.4) on a test set X test ⊆ C sampled from a distribution µ , it is possible to derive estimates of the L 2 -approximation error ∥ g -f ∗ ∥ 2 L 2 ( D,ν ) with respect to the push-forward ν = µ ◦ ( ∇ f ) -1 of µ under the gradient

map ∇ f : C → D . This is valuable beyond the framework of this paper since it makes it possible to evaluate the performance of any numerical method for convex conjugation.

The training and test sets X train , X test ⊆ C used for training and testing a candidate approximator g θ of the convex conjugate f ∗ can be generated by sampling from an arbitrary distribution µ on C . But the gradient map ∇ f : C → D is distorting C and transforms any finite subset X ⊆ C into ∇ f ( X ) ⊆ D . If one wants ∇ f ( X ) to consist of independent realizations of a desired target distribution ν on D = ∇ f ( C ) , X has to be sampled from a distribution µ whose push-forward µ ◦ ( ∇ f ) -1 is similar to ν . We do this by training a neural network h ϑ : D → R d that approximates a generalized inverse of ∇ f : C → D . Then target training and test sets Y train , Y test ⊆ D can be generated according to a desired distribution ν on D , and the sets X train = h ϑ ( Y train ) and X test = h ϑ ( Y test ) can be used for training and testing g θ .

The remainder of the paper is organized as follows: In Section 2, we introduce DLT. In Section 3, we derive estimates of the L 2 -approximation error of any candidate approximating function of f ∗ with respect to a given probability measure ν on the image D = ∇ f ( C ) of C under the gradient map ∇ f . In Section 4, we learn a generalized inverse mapping of ∇ f that improves the performance of DLT if ∇ f is heavily distorting C and is needed to estimate the L 2 -approximation error with respect to a desired target distribution ν on D . In Section 5, we report the results of different numerical experiments. Sec 6 concludes. Proofs of theoretical results and additional numerical experiments are given in the Appendix.

2 Convex conjugation as a machine learning problem

Let f : C → R be a differentiable convex function defined on an open convex subset C ⊆ R d . Our goal is to approximate the convex conjugate f ∗ : R d → ( -∞ , ∞ ] on the image D = ∇ f ( C ) of C under the gradient map ∇ f .

By the Fenchel-Young inequality 5 , one has

with equality if and only if y = ∇ f ( x ) . As a result, one obtains that on D , the convex conjugate f ∗ coincides with the Legendre transform

where the preimage ( ∇ f ) -1 ( y ) can be substituted by any x ∈ C satisfying ∇ f ( x ) = y . However, while in many situations, the function f and its gradient ∇ f are readily accessible, the inverse gradient map ( ∇ f ) -1 might not exist, or if it does, it is often not be available in closed form. Therefore, we use the implicit Legendre representation:

which makes it possible to formulate the numerical approximation of f ∗ as a machine learning problem. More precisely, for any machine learning model g θ : D → R with parameter θ in a set Θ , DLT approximates f ∗ on D ⊆ R d by minimizing the empirical squared error over θ ∈ Θ . In principle, X train can be any finite set of training data points x ∈ C . But in Section 4 below, we describe how it can be assembled such that the image ∇ f ( X train ) is approximately distributed according to a given target distribution ν on D = ∇ f ( C ) .

Any numerical solution ˆ θ ∈ Θ of the minimization problem (2.2) yields an approximation g ˆ θ : D → R of the true conjugate function f ∗ : D → R . The question is how accurate the approximation is.

3 A posteriori approximation error estimates

Thanks to the implicit form of the Legendre transform (2.1) it is possible to estimate the L 2 -approximation error of any candidate approximator g : D → R of f ∗ : D → R with a standard

Monte Carlo average even if the true f ∗ is not known. Let X test be a test set of random points x ∈ C independently sampled from a probability distribution µ over C . Then, due to (2.1), one has

which, by the law of large numbers, for |X test |→ ∞ , converges to

which, in turn, equals the squared L 2 -approximation error

with respect to the push-forward ν = µ ◦ ( ∇ f ) -1 of the measure µ . This result lets us report Monte Carlo estimates 6 of the error ∥ g -f ∗ ∥ 2 L 2 ( D,ν ) as if f ∗ were known in closed form.

4 Sampling from a given distribution in gradient space

The most straight-forward way to generate training and test sets X train , X test ⊆ C is to sample independent random points according to a convenient distribution µ on C .

But if one wants to minimize the L 2 -approximation error (3.1) with respect to a given distribution ν on D = ∇ f ( C ) , µ has to be chosen so that its push-forward µ ◦ ( ∇ f ) -1 equals ν . For instance, if the goal is a close approximation everywhere on D for a bounded subset D ⊆ R d , e.g. a ball or hypercube in R d , one might want ν to be the uniform distribution on D . If D is unbounded, e.g. D = R d , one might want ν to be a multivariate normal distribution. But if one is interested in a close approximation of f ∗ in a neighborhood of a particular point y 0 ∈ D , it is better to choose a distribution ν that is concentrated around y 0 .

To sample points x in C such that the distribution of ∇ f ( x ) is close to ν , we learn a generalized inverse of the gradient map ∇ f : C → D . This can be done by training a neural network h ϑ : D → R d with parameter ϑ ∈ R q such that either

We choose a standard feed-forward neural network h ϑ with two hidden layers and GELU-activation. To achieve (4.1a) or (4.1b), one can either minimize

for suitable training sets W train ⊆ C or Y train ⊆ D , respectively. (4.2a) and (4.2b) both have advantages and disadvantages. Therefore, we combine the two.

We first pretrain h ϑ by minimizing (4.2a). This works even if the initialized network h ϑ maps some points y from D to R d \ C . W train can e.g. be sampled from a uniform distribution over C if C is bounded or a multivariate normal distribution if C is unbounded. If ∇ f is not invertible, (4.2a) can typically not be reduced to zero. But pretraining will force h ϑ to map most of D into C . After pretraining, we minimize a convex combination of (4.2a) and (4.2b) for Y train independently sampled from ν . In every training step, points y ∈ Y train for which h ϑ ( y ) is not in C are omitted from training. The training procedure can be stopped if (4.2b) is sufficiently close to zero, yielding an approximate generalized inverse h ˆ ϑ of ∇ f . If Y test is a test set independently sampled of Y train according to ν , then X train = C ∩ h ˆ ϑ ( Y train ) and X test = C ∩ h ˆ ϑ ( Y test ) can be used to train and test a given machine learning model g ˆ θ to approximate f ∗ .

5 Numerical experiments

In this section, we report results of numerical approximations 7 of the convex conjugates of differentiable convex functions using DLT with MLPs, ResNets, ICNNs, and KANs. As an application, we use DLT to approximate solutions of Hamilton-Jacobi PDEs.

Numerical approximations with MLPs, ResNets and ICNNs

We first approximate f ∗ : D → R with different neural networks such as standard multi-layer perceptrons (MLPs), residual networks (ResNets) and input convex neural networks (ICNNs). We use MLPs with two hidden layers containing 128 units and GELU activation. Our ResNets have two residual blocks, each containing two dense layers, resulting in a 4-layer architecture with skip connections and GELU activation. The employed MLP-ICNNs have the same architecture as the MLPs but constrain hidden weights to be non-negative and apply softplus activations to ensure convexity. The ICNNs are specified as in Amos et al. [2017], using softplus activation and nonnegative weights in two hidden layers along with input skip connections. Parameters were initialized according to a Gaussian distribution, but for ICNN, the weights were squared to ensure non-negativity. For training, we used the Adam optimizer 8 .

As a first test, we compare DLT in the three cases of Table 1, where f ∗ is known analytically, to directly learning f ∗ using pairs ( y i , f ∗ ( y i )) as targets.

Table 2 shows that DLT matches the performance of direct learning both in terms of approximation accuracy and training time. This empirical finding is maintained across different network architectures and dimensions, confirming that the performance of DLT is only constrained by the neural networks' capacity to approximate multidimensional functions and not by its implicit formulation. In our experiments, ResNet provided the best approximations despite not guaranteeing convexity, while MLP-ICNN showed limitations due to the lack of skip connections compared to ICNN 9 .

Functions without closed form conjugate Most functions do not admit a closed form expression for the convex conjugate. However, DLT gives the same a posteriori error estimates as if f ∗ were known explicitly and learned directly on D . For illustration, we consider the quadratic over linear function

It is not defined everywhere on R d . But it can be shown that it is convex on the convex set C = { x ∈ R d : x 1 + · · · + x d > 0 } ; see e.g. Boyd and Vandenberghe [2004]. The i -th component of the gradient ∇ f ( x ) is

̸

But the convex conjugate f ∗ is not known in closed form, and the image D = ∇ f ( C ) is a non-trivial subset of R d . Even though we do not know f ∗ explicitly, DLT yields accurate ICNN approximations of f ∗ on D . Here, we used an ICNN with 3 hidden layers and ELU activations, we sampled random

x

2

1

0

1

2

3

4

3

2

1

0

1

2

3

4

2

Simulations from a 2-dim Half-Normal

20

15

10

5

0

25

20

15

10

5

0

x

2

Gradient-Transformed Sample Points

d

∥

g

θ

-

f

∗

∥

L

2

Time (mm:ss)

2

6.38e-04

4:14

5

2.39e-03

5:14

10

1.80e-02

6:55

20

3.67e-02

14:28

50

6.00e-02

29:19

100

3.68e-02

38:14

x

1

x

1

points from a standard half-normal distribution on C . Figure 1 depicts these random points and their images under the gradient map ∇ f given in (5.1) for d = 2 . The table on the right shows estimates of the L 2 -approximation error ∥ ∥ g ˆ θ -f ∗ ∥ ∥ L 2 ( R d ,ν ) together with the times in minutes and seconds it took to train the approximations g ˆ θ for different d . Note that the true f ∗ is unknown here, but due to the results of Section 3, it is still possible to provide L 2 approximation errors.

In Appendix B, we study the following additional non-separable examples: (i) quadratic f ( x ) = 1 2 x ⊤ Qx ; (ii) exponential-minus-linear f ( x ) = e ⟨ a,x ⟩ -⟨ b, x ⟩ ; and (iii) ICNN-represented functions.

Comparison with classical grid methods

Next, we compare DLT against the LLT algorithm of Lucet [1997], a state-of-the-art grid-based method for computing convex conjugates. Rather than directly evaluating max x ∈X {⟨ x, y ⟩ -f ( x ) } over a full d -dimensional grid X of size N d (which requires O ( N 2 d ) operations), Lucet reformulates the d -dimensional maximization as a sequence of nested 1D problems:

This decomposition reduces the computational complexity to O ( dN d +1 ) while still requiring O ( N d ) memory. Table 3 compares the methods for a fixed discretization of N = 10 points per dimension for two of the functions from Table 1. All errors were evaluated on the same random sample Y test ⊆ D = ∇ f ( C ) against the true f ∗ . For low dimensions ( d ∈ { 2 , 6 } ), Lucet runs quickly with modest memory, making it possible to use much finer grids within a reasonable time budget. However, as d grows beyond 8 , even modest grids become memory or time infeasible due to O ( N d )

storage and O ( d · N d +1 ) computational complexity. In contrast, DLT maintains a constant memory footprint and scales smoothly to dimensions d beyond 9.

Direct vs. inverse sampling: distorting gradient mapping

When the gradient mapping ∇ f is significantly distorting, the image ∇ f ( X ) of a uniformly sampled training set X ⊆ C will be highly non-uniformly distributed on D . This skewed distribution can adversely affect learning dynamics, concentrating approximation accuracy in certain regions while neglecting others. The approximate inverse sampling method introduced in Section 4 addresses this issue by allowing us to sample from any desired distribution (e.g., uniform) directly on D .

As an illustration, we consider the multidimensional negative logarithm f ( x ) = -∑ d i =1 log( x i ) on C = (10 -3 , 10 -1 ) d . With ∇ f ( x ) = ( -1 /x 1 , . . . , -1 /x d ) , the gradient map severely distorts C , transforming it to D = ∇ f ( C ) = ( -1000 , -10) d . The known closed-form of the convex conjugate is f ∗ ( y ) = -∑ d i =1 log( -y i ) -d .

First, we train with points sampled uniformly from C (direct sampling). Then, we compare with our inverse sampling approach from Section 4, using X train = h ˆ ϑ ( Y train ) where Y train is sampled uniformly from D and h ˆ ϑ approximates the inverse of ∇ f . Table 4 shows the L 2 approximation errors and training times for both methods.

The inverse sampling method consistently produces better approximations (1-2 orders of magnitude improvement for d ≥ 10 ) at the cost of increased training time. Figure 2 illustrates this improvement for d = 2 , where properly addressing the gradient distortion leads to a more uniform distribution of approximation accuracy across D .

Applications to Hamilton-Jacobi equations

The Hamilton-Jacobi (HJ) equation is a cornerstone of classical mechanics, optimal control theory, and mathematical physics, which provides a natural application domain for DLT.

Consider the Cauchy problem for the HJ equation:

for two convex functions H,g : R d → R .

The Hopf formula [Hopf, 1965] provides the following expression of u as a double convex conjugate:

Sampling

y

= (

y

1

, y

2

)

in

D

(

y, g

ˆ

θ

(

y

))

,

y

1

=

-

550

(

y, g

ˆ

θ

(

y

))

,

y

2

=

-

550

With Appr. Inv.

ScatterPlotofsamplinginD

g

g

11

-11

200

TrueConjugate

TrueConjugate

12

-12

=400

13

-13

600

14

-14

008=

=1000

-15

-15

1000

800

600

y1

400

002-

0

-1000

800

600

y1

400

200

-1000

800

600

y2

400

200

0

ScatterPlotofsamplinginD

NumericalandAnalyticalDLTalongaxis1

-11

g

-11

ge

-200

TrueConjugate

True Conjugate

12

-12

-400

13

-13

600

008=

-14

-14

-1000

-15

-15

1000

800

600

y1

400

200

0

1000

008-

009-

y1

400

0

1000

008-

600

y2

400

200

0

While this formula is explicit, evaluating it in high dimensions remains computationally challenging. To address this, we propose Time-DLT (Time-Parameterized Deep Legendre Transform) which approximates ( g ∗ + tH ) ∗ using DLT with a neural network u θ ( x, t ) with input variables x and t .

Experimental results. We study an HJ equation (5.2) with H ( x ) = g ( x ) = 1 2 ∑ d i =1 x 2 i . It has the explicit solution u ( x, t ) = ∑ d i =1 x 2 i 2(1+ t ) , which we can use a reference. We use Time-DLT to compute a solution on C = ( -a, a ) d × [0 , T ] for a = T = 2 and compare against the DGM (Deep Galerkin Method) of Sirignano and Spiliopoulos [2018], both implemented using the same ResNet architectures with two residual blocks. DGM minimizes the residual of the PDE on C directly, while Time-DLT approximates the Hopf formula (5.3). Table 5 shows that Time-DLT outperforms DGM in approximating the true solution across different dimensions and times (however, we used the explicit expression for g ∗ and just computed the outer Legendre transform).

Near t = 0 , DGM performs better by directly enforcing the initial condition. But for larger t , the Time-DLT solutions are closer to the correct ones 10 indicated by a DGM/DLT L 2 error ratio ρ > 1 . Note that DGM aims to minimize PDE residuals, but small PDE residuals do not always correspond to small approximation errors.

Symbolic regression with KANs

Kolmogorov-Arnold networks (KANs) have recently been introduced by Liu et al. [2025]. Inspired by the Kolmogorov-Arnold representation theorem, KANs hold great potential in approximation tasks requiring high precision. Moreover, in cases, where the Legendre transform has a closed form expression in terms of special functions, KANs can be employed to obtain the exact solution using symbolic regression. This is a significant advantage compared to numerical approximation since it uncovers the underlying mathematical structure. To illustrate symbolic regression with KANs, we consider the three functions of Table 1 in two dimensions.

In all three cases we performed a symbolic regression with KANs to minimize the empirical squared error (2.2) for a training set X train sampled uniformly from the square [1 , 10] 2 . We used polynomials up to the third order, exp and log as basis functions. As can be seen from the results in Table 6, KANs are able to find the correct symbolic expressions for the Legendre transforms of these functions with the right constants (up to rounding errors).

6 Conclusion, limitation and future work

In this paper, we have introduced DLT (Deep Legendre Transform), a deep learning method for computing convex conjugates of differentiable convex functions f : C → R defined on an open convex set C ⊆ R d . Our approach provides a posteriori approximation error estimates and scales effectively to high-dimensional settings, outperforming traditional numerical methods.

The limitation of DLT is that it needs a function f : C → R to be differentiable and convex. Moreover, it only approximates the convex conjugate f ∗ , which can be defined on all of R d , on the image D = ∇ f ( C ) , which is contained in, but in general not equal to, dom f ∗ = { y ∈ R d : f ∗ ( y ) < ∞ } .

The extension of the approach to non-differentiable convex functions f : C → R and approximation of f ∗ on dom f ∗ will be studied in future work.

Acknowledgments and Disclosure of Funding

This work was supported by the Swiss National Science Foundation under Grant No. 10003723. We are grateful to the anonymous reviewers for their constructive comments and suggestions.

References

D. Adeoye and A. Bemporad. Self-concordant smoothing for large-scale convex composite optimization, 2023. arXiv preprint arXiv:2309.01781.

Amos. On amortizing convex conjugates for optimal transport. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR) , 2023.

Amos, L. Xu, and J. Z. Kolter. Input convex neural networks. In Proceedings of the 34th International Conference on Machine Learning (ICML) , volume 70, pages 146-155. PMLR, 2017.

Blondel and V. Roulet. The elements of differentiable programming, 2024. URL https: //arxiv.org/abs/2403.14606 .

Boyd and L. Vandenberghe. Convex Optimization . Cambridge University Press, 2004.

Brenier. Un algorithme rapide pour le calcul de transformées de Legendre-Fenchel discrètes. C. R. Acad. Sci. Paris. Sér. I Math. , 308:587-589, 1989.

Chen, Y. Shi, and B. Zhang. Optimal control via neural networks: a convex approach. In Proceedings of the 7th International Conference on Learning Representations (ICML) , volume 7, 2019.

W. Cooley and J. W. Tukey. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation , 90:297-301, 1965.

Corrias. Fast Legendre-Fenchel transform and applications to Hamilton-Jacobi equations and conservation laws. SIAM Journal on Numerical Analysis , 33:1534-1558, 1996.

Dam, Q. Hoang, T. Le, T. Nguyen, H. Bui, and D. Phung. Three-player Wasserstein GAN via amortised duality. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (AAAI) , pages 2202-2208, 2019.

Fenchel. On conjugate convex functions. Canadian Journal of Mathematics , 1(1):73-77, 1949.

Haque and Y. Lucet. A linear-time algorithm to compute the conjugate of convex piecewise linear-quadratic bivariate functions. Computational Optimization and Applications , 70(2):593-613, 2018.

He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 770-778, 2016.

J.-B. Hiriart-Urruty. Lipschitz r -continuity of the approximative subdifferential of a convex function. Mathematica Scandinavica , 47:123-134, 1980.

Hopf. Generalized solutions of nonlinear equations of the first order. Journal of Mathematical Mechanics , 14:951-973, 1965.

P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2017.

Korotin, V. Egiazarian, A. Asadulaev, A. Safin, and E. Burnaev. Wasserstein-2 generative networks. In International Conference on Learning Representations (ICLR) , 2021a.

Korotin, L. Li, A. Genevay, J. Solomon, A. Filippov, and E. Burnaev. Do neural optimal transport solvers work? A continuous Wasserstein-2 benchmark. In Advances in Neural Information Processing Systems (NeurIPS) , 2021b.

A.-M. Legendre. Mémoire sur l'intégration de quelques équations aux différences partielles. Mémoires de l'Académie Royale des Sciences , 1787.

Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljacic, T. Y. Hou, and M. Tegmark. KAN: Kolmogorov-Arnold networks. In The Thirteenth International Conference on Learning Representations (ICLR) , 2025.

Lucet. Faster than the Fast Legendre Transform, the Linear-Time Legendre Transform. Numerical Algorithms , 16(2):171-185, 1997.

Lucet, H. H. Bauschke, and M. Trienis. The piecewise linear-quadratic model for computational convex analysis. Computational Optimization and Applications , 43(1):95-118, 2009.

Makkuva, A. Taghvaei, S. Oh, and J. Lee. Optimal transport mapping via input convex neural networks. In Proceedings of the 37th International Conference on Machine Learning (ICML) , volume 119 of Proceedings of Machine Learning Research , pages 6672-6681. PMLR, 2020.

Nesterov. Smooth minimization of non-smooth functions. Mathematical Programming , 103(1): 127-152, 2005.

Noullez and M. Vergassola. A fast Legendre transform algorithm and applications to the adhesion model. Journal of Scientific Computing , 9:259-281, 1994.

Peyré. The Legendre transform can be approximated in O(n log(n)) using a Gaussian convolution. X , 2020.

T. Rockafellar. Convex Analysis . Princeton Mathematical Series. Princeton University Press, Princeton, N. J., 1970.

A. Sharifi Kolarijani and P. Mohajerin Esfahani. Fast approximate dynamic programming for input-affine dynamics. IEEE Transactions on Automatic Control , 68(10):6315-6322, 2023.

Sirignano and K. Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics , 375:1339-1364, 2018.

Taghvaei and A. Jalali. 2-Wasserstein approximation via restricted convex potentials with application to improved training for GANs. arXiv preprint arXiv:1902.07197 , 2019.

A Unbiased a posteriori error estimator

Lemma A.1 ( Unbiased a posteriori error estimator). Let f : C → R be a differentiable convex function on an open convex set C ⊆ R d and denote D = ∇ f ( C ) . Fix a probability measure µ on C and consider the push-forward ν = µ ◦ ( ∇ f ) -1 on D . For a measurable function g : D → R , denote

where X 1 , . . . , X n are i.i.d. random vectors with distribution µ . Then the following hold:

(i) If g -f ∗ ∈ L 2 ( D,ν ) , one has

and ̂ E n ( g ) →∥ g -f ∗ ∥ 2 L 2 ( D,ν ) almost surely for n →∞ .

(ii) If g -f ∗ ∈ L 4 ( D,ν ) , then

and √ n ( ̂ E n ( g ) -∥ g -f ∗ ∥ 2 L 2 ( D,ν ) ) →N (0 , σ 2 ) in distribution as n →∞ . In particular, if ̂ σ is an estimate of σ and z α denotes the α -quantile of N (0 , 1) , then

is an approximate (1 -α ) -confidence interval for ∥ g -f ∗ ∥ 2 L 2 ( D,ν ) .

Proof. (i) By the Fenchel-Young equality f ∗ ( ∇ f ( x )) = ⟨ x, ∇ f ( x ) ⟩ -f ( x ) , one has

So, taking expectation and using the push-forward measure ν = µ ◦ ( ∇ f ) -1 yields

and it follows from the law of large numbers that ̂ E n ( g ) → ∥ g -f ∗ ∥ 2 L 2 ( D,ν ) almost surely. This shows (i).

(ii) Denote

Then ̂ E n ( g ) = 1 n ∑ n i =1 Z i , and therefore,

So, it follows from the central limit theorem that √ n ( ̂ E n ( g ) -∥ g -f ∗ ∥ 2 L 2 ( D,ν ) ) →N (0 , σ 2 ) in distribution. In particular, ̂ E n ( g ) ± ̂ σ √ n z 1 -α/ 2 is an approximate (1 -α ) -confidence interval for ∥ g -f ∗ ∥ 2 L 2 ( D,ν ) , which concludes the proof.

B Technical appendix: additional experiments

Content

Sections B.1 and B.2 describe low-dimensional experiments with grid-based methods: Lucet's Linear Time Legendre Transform and Peyré's Gaussian convolution approach with Monte Carlo integration.

Section B.3 presents comprehensive benchmarks comparing our DLT approach against direct supervised learning when analytical conjugates are known. Using identical neural architectures across multiple functions and dimensions d ∈ { 2 , . . . , 200 } , we demonstrate that DLT achieves equivalent L 2 errors without requiring closed-form Legendre transforms.

Section B.4 evaluates DLT on challenging non-separable convex functions. We test quadratic forms with random matrices, exponential-minus-linear functions, pretrained ICNNs, and coupled softplus functions with quadratic pairwise interactions. These experiments show that DLT maintains accuracy across strongly coupled functions, although training time increases due to the more complex computation tree required for gradient evaluation in non-separable functions.

Section B.5 compares DLT with an alternative algorithm that uses an approximate inverse gradient mapping instead of exact gradients for training. While DLT trains on exact targets using the FenchelYoung identity with true gradient values, the alternative method must learn an approximate inverse mapping to construct its training targets. We show that although such an approximation is feasible, it fundamentally depends on the quality of the inverse gradient approximation and provides weaker theoretical guarantees.

Section B.6 describes DLT applications to Hamilton-Jacobi PDEs via the Hopf formula. We introduce Time-Parameterized DLT that approximates ( g ∗ + tH ) ∗ and compare against Deep Galerkin Methods, showing superior accuracy.

Section B.7 provides additional experiments with approximate inverse sampling for targeting specific dual space regions.

Section B.8 explores KANs integration with DLT for symbolic regression tasks, investigating whether approximate inverse sampling improves symbolic expression recovery in low-dimensional settings.

Grid-based methods: definition-based method and Linear Time Legendre Transform

The definition-based method

The definition-based method implements convex conjugation directly according to the LegendreFenchel definition (1.1)

In explicit multi-dimensional form:

Key characteristics of the definition based method

Brute-force approach : The algorithm exhaustively evaluates all possible coordinate combinations for each slope combination.

Computational complexity : O ( N 2 d ) , where N is the number of grid points per dimension and d is the dimensionality. This complexity arises from:

O ( N d ) operations to iterate through all slope combinations

Memory usage : O ( N d ) for storing the output array, with minimal additional temporary storage.

O ( N d ) operations to search through all coordinate combinations for each slope

The definition based method provides a baseline implementation before considering more contemporary approaches like the Linear-time Legendre Transform (LLT).

The nested formulation

The nested method exploits the mathematical structure of the Legendre-Fenchel transform by reformulating it as a sequence of nested maximization problems. For a function f ( x ) in d dimensions, the transform can be rewritten in nested form

This decomposition allows dimension-by-dimension computation, starting from the innermost term.

Algorithm structure

1D Linear-time Legendre Transform (LLT) The LLT algorithm by Lucet [1997] efficiently computes f ∗ ( s ) in O ( n + m ) time, where n is the number of data points and m is the number of slopes:

Convex hull step : Compute the convex hull of the discrete data points ( x i , f ( x i )) .

Slope computation : Calculate slopes between consecutive vertices of the convex hull:

Merging step : For each target slope s j , find the interval [ c i , c i +1 ] containing s j , then compute:

d -dimensional algorithm The nested approach proceeds in d stages:

Compute the innermost term:

Work outward for i = d -1 down to 1 :

The final result is f ∗ ( s 1 , . . . , s d ) = V 1 ( s 1 , . . . , s d )

Comparative characteristics

Assuming equal grid resolution in primal and dual spaces ( n = m ):

Dimensional decomposition : The nested method decomposes the d -dimensional problem into d sequential lower-dimensional maximizations. The resulting performance (generally) depends on the order in which this decomposition is performed.

Computational complexity : O ( dN d +1 ) for the nested approach, compared to O ( N 2 d ) for the definition-based method. Each stage requires O ( N d +1 ) operations, with d total stages.

Memory usage : O ( N d ) for the nested method.

Error dependency : Transform accuracy for both methods depends on grid resolution and therefore identical.

Results. Each benchmark table reports performance and accuracy metrics for various input dimensions d , using uniform grids with n = m = 10 points per dimension 11 .

The column t Lucet (s) reports the runtime of the Nested Lucet method in seconds, while t Def (s) gives the runtime of the brute-force definition-based method (evaluated only for d ≤ 5 due to computational constraints). The columns MB L and MB D indicate the peak memory usage in megabytes (MB) during execution for the Lucet and Def methods, respectively; 0 . 0 reads that less than 0 . 05 MB was used. The column max | L -D | measures the maximum absolute difference between the Lucet and Def outputs across all grid points. Lastly, max err captures the interpolation error of the Lucet method relative to a reference solution obtained via uniform sampling, and RMSE reports the root mean squared error over the same validation sample. Some columns are indicated as - since direct computations are too expensive to run.

Generally, both methods struggle with the curse of dimensionality; however, Lucet's nested LLT method is more efficient than the direct computation on the grids.

The nested approach demonstrates substantial computational advantages as dimensionality increases, albeit at the expense of increased memory requirements. However, despite the sparse grid used, it becomes intractable for d ≥ 10 .

Softmax-Based approximation of Legendre-Fenchel transforms

The softmax-based approach, as proposed by Peyré, see Peyré [2020] and Blondel and Roulet [2024], offers an alternative method for approximating Legendre-Fenchel transforms through a convolution-based formulation. For a convex function f defined on a bounded domain C ⊆ R d , the Legendre-Fenchel transform can be approximated using:

where ε > 0 is a smoothing parameter that controls the accuracy of the approximation. As ε → 0 , the softmax approximation f ∗ ε converges to the exact transform f ∗ .

This formulation can be rewritten in terms of Gaussian convolution:

where: G ε ( x ) = exp( -1 2 ε ∥ x ∥ 2 2 ) is a Gaussian kernel, Q ε ( f )( x ) = exp( 1 2 ε ∥ x ∥ 2 2 -1 ε f ( x )) and Q -1 ε ( F )( s ) = 1 2 ∥ s ∥ 2 2 -ε log F ( s ) .

Implementation approaches

Three primary techniques were employed:

Direct integration: For low dimensions ( d ≤ 2 ), the integral is directly approximated on a grid.

FFT-based convolution: For higher dimensions, the method uses the Fast Fourier Transform to compute the convolution efficiently, with complexity O ( dN d log N ) .

(Quasi) Monte Carlo: to approximate integrals. More memory efficient than integration but still expensive to iterate through dual grids or samples { y i } .

Empirical results

In our experiments Monte Carlo approach showed a more stable results for performing integration for small values of ε , so we focus on it. Our MC implementation uses Sobol sequences 12 to generate low-discrepancy sample points, achieving better coverage of the integration domain compared to purely random sampling. Table B5 presents results for the negative logarithm function using 100,000 samples. The dual grid is the same as in LLT experiments from the previous section, see Table B3.

Experiments confirm O ( ε ) convergence to the exact Legendre transform as ε → 0 . Peyré's approach is suitable when an estimate of the transform is required at only one point y in the dual space, but it is still too expensive to iterate over dual grids/samples, especially in high-dimensional setups: d > 4 .

Our experiments showed that the method performs better for smooth functions such as negative entropy, and less effectively for functions with steep gradients like the negative logarithm.

DLT: Comparing different NN architectures

Comparison with direct learning when the Legendre transform is known

A critical validation of DLT is to demonstrate that it achieves state-of-the-art performance by comparing it to the direct learning of the convex conjugate in cases where the analytic form of f ∗ is already known. For this comparison, we implemented a comprehensive benchmark that compares DLT 13 against explicit direct supervised learning across several convex functions with known convex conjugates in multiple dimensions.

Benchmark setup

We compare two learning approaches:

DLT: trains a neural network g θ to satisfy g θ ( ∇ f ( x )) ≈ ⟨ x, ∇ f ( x ) ⟩ -f ( x ) .

Direct: directly trains g θ so that g θ ( y ) ≈ f ∗ ( y ) , which is only possible if f ∗ is known explicitly.

The direct approach serves as our 'gold standard" since it has direct access to the target function values. However, this advantage is only available in the special cases where f ∗ has a known analytical expression-an assumption that does not hold in many practical applications. Our goal is to demonstrate that DLT matches the performance of this idealized case.

We implemented both approaches using identical neural network architectures to ensure a fair comparison:

MLP: Standard multi-layer perceptron with two hidden layers (128 units each) using ReLU activations

MLP-ICNN: Feed-forward network with enforced convexity using only positive weights (two hidden layers with 128 units each)

ResNet: Residual network with two residual blocks (128 units each)

ICNN: Input-convex neural network of Amos et al. [2017] with positive z-weights and direct x-skip connections (two hidden layers with 128 units each)

The implementation uses JAX for efficient computation with identical optimization settings for both approaches: Adam optimizer with learning rate 10 -3 and batch size 128 × d . We employed an early stopping criterion where training terminates if the L 2 2 error falls below 10 -6 , with a maximum limit of 50,000 iterations to ensure computational feasibility across all experiments. To estimate statistical significance, we use 10 runs and report both the mean ρ and standard deviation σ of residuals achieved with direct and indirect methods.

A key consideration in our benchmark design was ensuring that sampling distributions are correctly matched between primal and dual spaces. The gradient map ∇ f : C → D transforms the distribution of points x ∈ C into a distribution over y = ∇ f ( x ) ∈ D . For each function, we designed specific sampling strategies:

Quadratic f ( x ) = ∥ x ∥ 2 2 2 : Since ∇ f ( x ) = x , we sample x ∼ N (0 , I ) for the implicit approach, which naturally gives y = x ∼ N (0 , I ) in the dual space for the explicit approach.

Negative logarithm f ( x ) = -∑ d i =1 log( x i ) : For the implicit approach, we sample x uniformly in log-space from (0 . 1 , 10) d to adequately cover the positive domain. Since ∇ f ( x ) = -1 x , this transforms to y ∈ ( -10 , -0 . 1) d in dual space. For the explicit approach, we directly sample y from this same distribution.

Negative entropy f ( x ) = ∑ d i =1 x i log( x i ) : Similarly, we sample x uniformly in log-space for the implicit approach. With gradient ∇ f ( x ) = log( x )+1 , this maps to y ∈ ( -1 . 3 , 3 . 3) d in dual space, which we match in our explicit approach sampling.

This distribution matching is essential for a fair comparison, as it ensures both approaches face comparable learning challenges.

Results and analysis We tested these three convex functions with known analytic conjugates across dimensions d ∈ { 2 , 5 , 10 , 20 } :

The quadratic function f ( x ) = ∥ x ∥ 2 2 / 2 is particularly elegant for our comparison since it is selfconjugate ( f ∗ = f ) and has gradient ∇ f ( x ) = x . This means that when using DLT with samples x ∼ N (0 , I ) , we get y = ∇ f ( x ) = x ∼ N (0 , I ) in the dual space as well, making the comparison especially clean.

Table B7 presents L 2 errors and training times for both approaches. These benchmark results reveal a key finding: DLT matches the performance of direct learning if the convex conjugate is known analytically. The L 2 errors for both methods are statistically identical .

When the dual validation samples are drawn from the training distribution, DLT learns the convex conjugate just as accurately (and almost as fast) as direct supervised learning of f ∗ -even though it never sees the true dual values.

These results validate a key advantage of our approach: DLT achieves the same approximation accuracy as methods that require knowing the analytical form of f ∗ , without actually needing this knowledge. This makes DLT applicable to the wide range of problems where f ∗ does not have a known closed form, while still maintaining state-of-the-art performance in cases where the Legendre transform is known.

Experiments with non-separable functions

To demonstrate DLT's capability beyond simple separable functions, we evaluate its performance on several challenging non-separable convex functions. Table B10 presents results across four representative function classes:

Quadratic form with random SPD matrix : We test f ( x ) = 1 2 x T Qx where Q is a randomly generated symmetric positive definite matrix with controlled condition number κ ≈ 10 2 . The coupling through Q makes this strongly non-separable, requiring the approximator to learn the full quadratic form structure.

Exponential-minus-linear : The function f ( x ) = exp( ⟨ a, x ⟩ ) - ⟨ b, x ⟩ where a, b ∈ S d -1 are independently sampled unit vectors. Specifically, we sample a raw , b raw ∼ N (0 , I d ) and normalize them as a = a raw / ∥ a raw ∥ 2 and b = b raw / ∥ b raw ∥ 2 .

Pretrained ICNN : We first train a 2-layer ICNN (128 hidden units per layer) for 50,000 iterations to approximate a target quadratic function, then freeze it and use it as the convex function f . This represents a realistic scenario where the target function itself is a learned model from data.

Coupled Soft-plus : The function f ( x ) = ∑ i<j log(1 + exp( x i + x j )) explicitly couples all pairs of variables with O ( d 2 ) interaction terms. This severely challenges classical methods due to the combinatorial explosion of terms.

Experimental setup. All experiments employ ResNet approximators with 2 blocks of 128 hidden units each, trained using the DLT framework. We train for 120,000 iterations with a learning rate of 10 -3 and Huber loss ( δ = 1 . 0 ) to ensure robust convergence. For each function class, we apply domain-specific whitening transformations. Training samples x are drawn from appropriate domains

with truncated normal or uniform distribution (see Table B9), with batch sizes adaptively set to 25% of the dimension ( B = ⌊ 0 . 25 d ⌋ ). We evaluate performance at dimensions d ∈ { 10 , 20 , 50 } and report results averaged over 5 independent trials with different random seeds.

Results. The results demonstrate that DLT successfully learns all four non-separable function classes, achieving low RMSE errors across different dimensions. The quadratic SPD function shows the best performance with errors decreasing as dimension increases (from 1 . 15 × 10 -2 at d = 10 to 2 . 01 × 10 -3 at d = 50 ), demonstrating DLT's effectiveness in learning structured covariance patterns. The exponential-minus-linear function presents greater challenges due to its unbounded gradient domain, yet DLT maintains reasonable accuracy. The pretrained ICNN case validates DLT's ability to approximate learned neural network functions.

Notably, the coupled soft-plus function requires approximately 2 × longer training time at d = 20 and d = 50 due to its O ( d 2 ) pairwise interaction terms and longer gradient evaluation step. The relative RMSE remains below 2% for most cases, confirming that DLT provides accurate approximations suitable for downstream optimization tasks. These experiments establish that DLT with simple ResNet architectures can effectively handle diverse non-separable convex functions that would be intractable for classical decomposition methods.

Alternative NN approaches to computing Legendre transforms

DLT approximates the convex conjugate f ∗ on D = ∇ f ( C ) by leveraging the exact Fenchel-Young identity at gradient-mapped points. The key insight is that for any x ∈ C with y = ∇ f ( x ) , the identity f ∗ ( y ) = ⟨ x, y ⟩ -f ( x ) holds exactly.

±

We train a neural network g θ : D → R by minimizing the loss:

where x ∈ C is sampled to ensure y = ∇ f ( x ) adequately covers D . Crucially, this loss equals ( g θ ( y ) -f ∗ ( y )) 2 for points on the manifold y = ∇ f ( x ) , providing an immediate accuracy certificate through the training residual itself.

±

±

In contrast, one can consider a proxy method by using an approximate target:

This introduces potential bias since h ϑ is only an approximation of the true inverse gradient. More precisely, if a model is trained to the proxy target t h ( y ) = ⟨ h ϑ ( y ) , y ⟩ -f (( y )) , then

Even if optimization drives the first term down, the certificate still reflects the second term, which is zero only when h ϑ = ( ∇ f ) -1 on D . While proxy methods can achieve low training loss on their approximate targets, we evaluate all methods using our DLT certificate (DLT), which measures the true approximation error regardless of the training approach used.

DLT provides several critical advantages over the Proxy method:

Guaranteed convexity : When using an ICNN architecture, DLT guarantees the convexity of the output g θ since the training target is the true convex conjugate f ∗ . The Proxy method

loses this guarantee because even with an ICNN, the approximator learns a biased target ⟨ h ϑ ( y ) , y ⟩ -f ( h ϑ ( y )) where h ϑ may not be the true inverse gradient.

No accuracy bottleneck : DLT's accuracy is limited only by the neural network's approximation capacity, not by an intermediate operator approximation. The Proxy method's final accuracy is fundamentally bottlenecked by the precision of h ϑ ≈ ( ∇ f ) -1 , regardless of the main network's capacity.

Efficient sampling : DLT uses the exact gradient mapping ∇ f for sampling, which is computationally efficient and exact. The Proxy method requires every new sample to pass through the learned inverse network h ϑ , adding computational overhead and potential approximation errors at inference time.

Training overhead : While both methods may use an inverse mapping for sampling control, the Proxy method critically depends on the quality of this mapping for its training targets, requiring more careful and time-consuming pretraining.

Table B11 reports numerical results 14 for the functions from Table B4, comparing DLT with approximate-inverse sampling with the inverse gradient proxy baseline for dimensions d ∈ { 5 , 10 } . Both methods share the same inverse pretraining time t h , which ranges from approximately 4 to 8 minutes depending on the function and dimension. We learn the inverse mapping h ϑ to high precision ( h ϑ RMSE between 10 -6 and 10 -4 ). As a result, the certificate RMSE-which measures the true approximation error of f ∗ -becomes comparable in terms of training times between the two methods.

Handling out-of-domain inverse mappings. When using an approximate inverse mapping h ϑ : D → R d for sampling, some points h ϑ ( y ) may fall outside the constraint set C . Two approaches exist for handling these cases: (i) adding a penalty term to the loss function that penalizes violations of h ϑ ( y ) / ∈ C , or (ii) discarding such samples from the training batch. While the penalty approach could

theoretically guide the network toward valid mappings, it introduces additional hyperparameters (penalty weight, barrier function choice) and may distort the training dynamics by mixing two objectives-approximating the inverse and satisfying constraints. We adopt the discarding approach as it maintains the purity of the training objective.

Hamilton-Jacobi equation and time-parameterized inverse gradient sampling

The Hamilton-Jacobi (HJ) equation is a cornerstone of classical mechanics, optimal control theory, and mathematical physics. It is given by the first-order nonlinear partial differential equation

where u ( x, t ) represents the action (value) function, and H denotes the Hamiltonian associated with a physical system or control problem.

Consider the Cauchy problem:

where Q = R d × (0 , ∞ ) .

The Hopf formula and function definitions

The Hopf formula provides an explicit solution to the Hamilton-Jacobi equation:

For our computational experiments, we consider two classes of functions that exhibit different mathematical properties and computational challenges.

Quadratic functions. These represent the classical case with explicit analytical solutions:

Initial condition: g ( x ) = 1 2 ∑ d i =1 x 2 i

Conjugate: g ∗ ( p ) = 1 2 ∑ d i =1 p 2 i

Hamiltonian: H ( p ) = 1 2 ∑ d i =1 p 2 i

In this case, the analytical solution is u ( x, t ) = ∑ d i =1 x 2 i 2(1+ t ) .

Exponential functions. These provide a more challenging test case:

Initial condition: g ( x ) = ∑ d i =1 e x i

Hamiltonian: H ( p ) = ∑ d i =1 e p i

Conjugate: g ∗ ( p ) = ∑ d i =1 ( p i log p i -p i ) (when p i > 0 )

To approximate the HJ solution u ( x, t ) we use a time-parameterized network u θ ( x, t ) with input ( x, t ) .

Time-parameterized inverse gradient sampling

Denote v ( p, t ) := g ∗ ( p ) + t H ( p ) . Then the Hopf representation can be written as

Whenever v ( · , t ) is differentiable (and, e.g., strictly convex so the maximizer is unique), the maximizer p = p ( x, t ) is characterized by the first-order optimality condition

Time-DLT therefore requires samples of pairs ( x, p ) satisfying this relation. In the case where H and g are quadratic one has ∇ p v ( p, t ) = (1 + t ) p , so x = (1 + t ) p (equivalently p = x/ (1 + t ) ), which allows direct sampling. In non-quadratic cases the inverse map ( ∇ p v ( · , t )) -1 is not available in closed form; to generate samples with a prescribed distribution in x -space we learn a timeparameterized inverse-gradient network Ψ ϕ ( x, t ) ≈ ( ∇ p v ( · , t )) -1 ( x ) . Similar to Section 5 we train Ψ ϕ using ∥∇ p v (Ψ ϕ ( x, t ) , t ) -x ∥ 2 2 and ∥ Ψ ϕ ( ∇ p v ( p, t ) , t ) -p ∥ 2 2 as loss funcitons, which only require evaluating ∇ p v = ∇ g ∗ + t ∇ H . In the implementation, this provides the inverse-gradient sampling step p ≈ Ψ ϕ ( x, t ) (and in the quadratic case the explicit formula p = x/ (1+ t ) ) used to form training triplets ( x, p, t ) .

Time-DLT implementation

Time-DLT directly approximates the Hopf formula through with neural network u θ ( x, t ) that learns:

The training process minimizes the Legendre transform residual:

The sampling of triplets ( x, p, t ) depends on the function type: analytical formulas for quadratic functions and the trained inverse network for exponential functions. This approach preserves the structure of the Hopf formula while ensuring computational tractability.

Experimental framework

We evaluate both methods across increasing dimensions and function complexities. We sample ( x, t ) ∼ Unif ( [ x min , x max ] d × [0 , t max ] ) , with x min = -2 , x max = 2 , t max = 3 and report in Tables B12 and B13 the resulted RMSE and PDE residuals across different time moments t ∈ { 0 . 1 , 0 . 5 , 1 . 0 , 2 . 0 } .

Problem configurations. We evaluate both methods across the following configurations:

Dimensions: d ∈ { 2 , 5 , 10 } to assess scalability

Function combinations:

-Quadratic-Quadratic: baseline with known analytical solution

-Quadratic-Exponential: Hamiltonian H is exponential

-Exponential-Quadratic: initial condition g is exponential

Neural network architecture Both methods employ time-parameterized multilayer perceptrons (MLPs):

Input:

( x, t ) ∈ R d × R +

Architecture: ( d +1) → 64 → 64 → 1

Activation: tanh

Optimizer: Adam with learning rate η = 10 - 3

For Time-DLT, the inverse gradient network h ϕ uses:

Input: ( p, t ) ∈ R d × R +

Architecture: ( d +1) → 64 → 64 → d

Two-phase training: 40% of epochs for , 60% for

h ϕ F θ

Implementation details

Ground truth computation. To assess accuracy when an explicit solution is unavailable, we compute reference solutions by evaluating for each point ( x, t ) a reference value by solving the inner optimization over p in the Hopf representation (B1):

Low dimensions ( d ≤ 2 ): Scipy's L-BFGS-B algorithm

Higher dimensions ( 2 < d ≤ 5 ): PyTorch gradient-based optimization with Adam

Training specifications.

Batch size: 100 samples per iteration

Training epochs:

1500 for d = 2 , scaling to 3000 for d = 10

Sampling strategy:

Adaptive mixing of initial condition and interior points

Regularization: λ = 100 for initial condition enforcement

Gradient clipping: Maximum norm of 10.0 for stability

Evaluation metrics

We assess performance using three complementary metrics:

L 2 error against ground truth:

The results demonstrate several key findings:

Accuracy: Time-DLT consistently achieves significantly lower errors compared to DGM across all problem configurations.

Scaling with dimension: Both methods show increased errors as dimension increases, but Time-DLT maintains superior performance even in higher dimensions.

The Time-DLT approach demonstrates that by explicitly modeling the time-parameterized convex conjugate ( g ∗ + tH ) ∗ , we can achieve more accurate and efficient solutions compared to direct PDE approximation methods. This approach is particularly effective for high-dimensional problems where g ∗ is known in closed form. If DLT is used to compute g ∗ as well, errors from such a double Legendre transform might accumulate, potentially degrading overall approximation accuracy.

PDE residual:

Initial condition error:

Notes: L 2 Error represents the mean squared difference between predicted solutions and the analytical solution u ( x, t ) = ∑ x 2 i 2(1+ t ) . HJ Error measures the residual of the Hamilton-Jacobi equation | ∂ t u + H ( ∇ x u ) | .

Notes: L 2 Error quantifies the mean squared deviation from the ground truth solution. PDE Residual is the residual of the Hamilton-Jacobi equation | ∂ t u + H ( ∇ x u ) | .

Comparison at t=0.1

d=2,H=exponential,IC=quadratic

Proper Time-DLT Solution

|DGM - Time-DLT|

2.0

2.0

3.6

2.0

0.600

3.5

1.5 -

1.5 -

3.2

1.5

0.525

1.0

3.0

1.0 -

2.8

1.0

0.450

2.5

2.4

0.5 -

0.5

S0

0.375

2.0

2.0

义

00

1.5

义

0.0

1.6

义

0°0

0.300

0.5 -

0.5 -

1.2

0.5

0.225

1.0

1.0

1.0 -

0.8

1.0 -

0.150

0.5

0.4

1.5 -

1.5

1.5 -

0.075

0.0

0.0

2.0

2.0

2.0 -

0.000

Comparison at t=0.5

d=2,H=exponential,IC=quadratic

Proper Time-DLT Solution

[DGM - Time-DLT]

2.0

3.6

2.0

3.5

2.0

0.32

1.5

3.0

1.5 -

3.0

1.5 -

0.28

1.0 -

2.4

1.0

2.5

1.0

0.24

0.5 -

1.8

0.5

2.0

1.5

0.20

义

00

1.2

义

0.0

义

0.0-

1.0

0.16

s'0-

0.6

0.5

0.5

s'0-

0.12

1.0 -

0.0

1.0

0.0

1.0

0.08

1.5 -

0.6

1.5 -

0.5

1.5

0.04

1.2

-1.0

2.0 +

2.0 +

2.0 +

0.00

Comparison at t=1.0

d=2,H=exponential,IC=quadratic

Proper Time-DLT Solution

[DGM - Time-DLT]

2.0

3.0

2.0

3.6

2.0

0.72

1.5 -

2.4

1.5 -

3.0

1.5

0.63

1.0 -

1.8

1.0

2.4

1.0 -

0.54

1.2

1.8

0.5 -

0.5 -

0.5

0.6

1.2

0.45

义

0°0

0.0

义

0°0

0.6

义

0°0

0.36

0.5

-0.6

0.5

0.0

0.5

0.27

1.0 -

1.2

1.0

-0.6

1.0

0.18

1.5 -

1.8

1.5 -

1.2

1.5 -

0.09

2.0

2.4

2.0

1.8

2.0

0.00

0

2

-1

0

X1

Approximate inverse sampling: additional experiments

In this section, we consider two more examples of approximate inverse sampling in the context of the Deep Legendre Transform.

The first example, when approximate inverse sampling might be particularly useful, is when the target distribution ν is concentrated around certain points { y i } n i =1 ∈ D of particular interest. Then, on the one hand, one has to sample x from a distribution µ , such that ∇ f ◦ µ has high density near all the points { y i } n i =1 ∈ D . On the other hand, if the points are distant enough, it might be inefficient to try to learn f ∗ on the whole D = ∇ f ( C ) given that we are interested in higher precision in the neighborhood of { y i } n i =1 ∈ D . We illustrate that approximate inverse sampling can solve both problems, efficiently sampling (approximately) to the target distribution ν.

In the second example, we explore the ability of KANs to find symbolic representations in lowdimensional setups with and without approximate inverse sampling. The setup is similar to that of Section 6.1 of the main part. Surprisingly, KANs managed to find correct symbolic expression even with a quite skewed sample.

The case of a target distribution concentrated around specific points

We consider the case of the multidimensional negative entropy

defined on the positive orthant C = { x ∈ R d : x i > 0 for all i } . It is straightforward to check that f ∗ ( x ) = ∑ d exp( x 1) .

Consider two points in R d : a point y with coordinates y i = 1 , for all i , and a point z with all coordinates equal to 4 . The target distribution ν is modeled as a fair mixture of normal distributions N ( y, 10 -3 I d ) and N ( z, 10 -3 I d ) .

Figures B3 and B3 illustrate the process of learning the inverse gradient mapping in this setup.

Epoch 7ooo1 - Direct vs Inverse sampling

120

100

80

X

60

40

20

μsamples

g-(v) samples

0

20

40

60

80

100

X1

2.50

2.5

2.5

2.25

2.00

2.0

2.0

Z tuauods

1.75

1.50

Component 2 of

>

1.5

1.0

Component 2 of

1.5

1.25

1.00

1.0

0.75

0.0

0.50

Nu samgling

Approximate inverse sampling

Nu sampling

Approximate inverse sampling

0.5

fugdues nN

Approximate inverse sampling

0.5

Nu sampling

Approximate inverse sampling

0.5

1.0

Component 1 of y

1.5

2.0

4

Component 1 of y

2

0

2

0.5

0.0

Component 1 of y

1.0

1.5

2.0

2.5

0.0

S°0

1.0

Component 1 of y

1.5

2.0

2.5

Kolmogorov-Arnold neural networks and approximate inverse sampling

Inspired by the Kolmogorov-Arnold representation theorem, which shows that every continuous function ϕ : [0 , 1] d → R can be written as

for one-dimensional functions ϕ i,j : [0 , 1] → R and Φ i : R → R , a KAN is a composition of KAN-layers of the form

where the one-dimensional functions ϕ l i,j : R → R are specified as

for given basis functions B i and learnable parameters c i ∈ R .

The canonical choice of B i are polynomial splines with learnable grid-points. But, depending on the situation, some of the B ′ i s can also be specified as special functions such as exp , log , sin , cos , tan .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly and accurately state the main contributions and scope of the paper, including the derivation of a posteriori error estimates, which are supported by the results in Sections 5 and the main theoretical result in Section 3.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper explicitly discusses limitations, such as the focus solely on differentiable convex functions, and does not address non-convex or non-differentiable scenarios.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper clearly states the assumptions required for the main theoretical result in Section 3, including differentiability and convexity. The proof is clearly presented and fully justified by invoking standard results such as law of large numbers.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Sections 5.1, 5.2, and the supplementary materials fully disclose model architectures, data distributions, training details, and evaluation metrics needed to reproduce the experimental results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The paper explicitly provides access to the code, data, and sufficient instructions in the supplementary materials to reproduce all experiments.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Experimental settings, including network architectures, activation functions, optimizers, data splits, and sampling methods, are specified in detail in Section 5 and supplementary materials.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper includes appropriate error bars and statistical significance analysis for the main numerical results, as detailed in the supplementary materials.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Section 5 and supplementary materials provide comprehensive details regarding GPU and CPU resources utilized, memory requirements, and explicitly report training times for all numerical experiments.

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research presented aligns fully with the NeurIPS Code of Ethics, involving no ethical conflicts or violations.

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper primarily presents methodological advancements in numerical methods without direct societal implications.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper does not involve high-risk data or models requiring safeguards.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: No external code, data, or assets are explicitly mentioned or used in the paper.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce new datasets, models, or assets.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing or human subjects research.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The research does not involve human subjects or IRB-related matters.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research?

Answer: [NA]

Justification: The research does not use large language models or similar tools in the core methodology.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors train a neural network to approximate the convex conjugate (Legendre–Fenchel transformation) of convex differentiable functions. Indeed, the convex conjugate $ f^* $ of $ f $ satisfies in this case $f^* (\\nabla f(x)) = \\langle x, \\nabla f(x) \\rangle - f(x)$, and therefore they minimise the distance between the output of the neural network and the expression $\\langle x, \\nabla f(x) \\rangle - f(x)$. However, this approximates the convex conjugate on $D = \\nabla f(C)$ only, where $C$ is the domain of $f$, and uniformly sampling training points in $C$ before transforming them through $\\nabla f$ sometimes leads to an uneven distribution over $D$. The authors address this second point by learning another neural network to approximate an inverse of $\\nabla f$, which allows to sample uniformly from $D$ and to then go back to $C$. However, $D$ itself is not always explicitly known as in the example in Figure 1 in the paper, which makes this solution impossible in this case, and makes it hard to know how much of the domain of $f^* $ we are covering.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Well-written\n2. Theoretically sound\n3. Multiple architectures were tested for the approximating neural network\n\nWeaknesses:\n1. The method is not compared to any other method (except in the case of its application to the HJ equation, and to a naive learning of a neural network to directly approximate the conjugate when it is available is closed form). The authors mention (and test in the supplementary material) some existing non-deep learning methods for approximating convex conjugates, but do not compare them to their method. These numerical approximation methods do not scale to high dimensions, but they could have tested their method in lower dimensions to compare.\n2. Even though the fact that their method only learns an approximation of the convex conjugate on $D = \\nabla f(C)$ is acknowledged, I think that this point warranted more discussion. How big is this set compared to the domain of $ f^* $ in typical/useful cases? How does the learned neural network approximate $ f^* $ outside of $D$ anyway?\n3. A few more tests and comparisons on problems such as the HJ equation where convex conjugates appear could have been included.\n\nQUESTIONS:\nSee point 2 in weaknesses.\n\nLIMITATIONS:\nWeakness 2 could be discussed further", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel deep learning approach for approximating the convex conjugates of differentiable convex functions. The core algorithm leverages an efficient gradient-based framework to minimize approximation error, utilizing the implicit Fenchel formulation of convex conjugation. The numerical experiments show the effectiveness of the proposed method. Besides that, the implicit Fenchel formulation also facilitates a posteriori approximation guarantees.\n\nSTRENGTHS AND WEAKNESSES:\n**Summary of Claims and Evaluation**\n\n**Claim 1**: Training a machine learning model to learn the convex conjugate f* yields strong numerical performance compared to directly learning f* from its analytic form.\n\nThis claim is supported by Table 2, which shows that the L2-approximation error of the proposed DLT method is comparable to that of direct learning using the analytic expression of f*. This also serves as an informative ablation study for evaluating the suitability of different neural network architectures and validates the proposed learning formulation (1.5).\n\n**Claim 2**: Using ICNN or ResNet as the reparameterization network enables scaling to high-dimensional problems and ensures that the resulting approximation of f* is convex.\n\nI have two concerns regarding this claim:\n- The experiments only go up to 50 dimensions (e.g., as shown in Table 1), which may not be sufficiently high to convincingly support claims about scalability to high dimensions.\n\n- Line 94 states that convexity is not guaranteed when using ResNet, which appears to contradict Claim 2’s assertion that the approximation is automatically convex. This inconsistency should be clarified.\n\n\n**Claim 3**: KAN (Kernel Activation Network) used as the reparameterization network can recover exact solutions for certain convex functions.\n\nThis claim is supported by the results in Table 5. However, the experiments are limited to low-dimensional (e.g., 2D) problems, which may limit its practicality. \n\n**Claim 4**: The implicit Fenchel formulation enables a posteriori approximation error estimation.\n\nThis claim is supported by the main theoretical result in Section 3. The result appears sound, although it falls outside my current area of expertise, so I cannot fully assess its correctness.\n\n**Strengths**\n\n- The paper is clearly written and well organized.\n\n- The authors' claims are largely supported by numerical results.\n\n- The experimental evaluations are extensive and clearly presented.\n\n**Weaknesses**\n\n- The proposed method is introduced somewhat abruptly in both the abstract and introduction. I suggest expanding the first paragraph of the introduction to better motivate the significance and applications of computing convex conjugates.\n\n- The experiments only go up to 50-100 dimensions (e.g., as shown in Table 1), which may not be sufficiently high to convincingly support claims about scalability to high dimensions.\n\n- Line 94 states that convexity is not guaranteed when using ResNet, which appears to contradict Claim 2’s assertion that the approximation is automatically convex. This inconsistency should be clarified.\n\n- The KAN experiments in Table 5 are limited to low-dimensional (e.g., 2D) problems, which may limit the practicality. Please consider elaborating on how this approach might generalize to higher dimensions.\n\n- Minor comment: It would also be helpful to report uncertainties for metrics such as approximation errors and computation time, especially since the latter can vary significantly across runs due to hardware-level fluctuations.\n\nQUESTIONS:\nPlease refer to **Strengths And Weaknesses** for my questions.\n\nLIMITATIONS:\nPlease refer to **Strengths And Weaknesses** for the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors consider a problem of finding convex conjugate for differentiable convex function. To construct approximation they propose to use neural network as parametrisation for convex conjugate and loss function that is minimised by stochastic gradient descent. Loss function is chosen such that for some convex functions approximation of inverse Jacobian matrix is required to adjust distribution on domain of convex function of interest. This approximation is achieved by a second neural network. Authors evaluate their method on several problems and find it to be competitive to other approaches.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Ideas proposed by authors are easy to understand\n2. Exhaustive context on the computation of convex conjugate is available which makes the article suitable for general audience\n3. Appendix contain thorough comparison with classical methods\n\n**Weaknesses**\n1. Novelty and relevance of the method are questionable\n2. Method lacks a posteriori approximation guarantees despite the claim of the authors\n3. Problems used for benchmarks are too simple\n\nSee below for details.\n\nQUESTIONS:\n1. **novelty and relevance**\n\n   Between lines 66 and 67 right before equation (1.4) authors formulate a natural loss that was already considered in https://arxiv.org/abs/2210.12153 for different sets of problems. Authors consider loss function (1.4) which is different from https://arxiv.org/abs/2210.12153 and apparently require to also approximate the inverse Jacobian matrix in practical cases. Essentially the primal proposition by the authors is to use a different loss function from https://arxiv.org/abs/2210.12153. In my view this significantly undermines the novelty of the contribution. It would be possible to consider the article from the point of view of applications, but only toy problems are chosen for evaluation (see below). Given that, I would like to ask the following questions:\n\n   a. Can the authors please defend the novelty of their approach in light of https://arxiv.org/abs/2210.12153?\n\n   b. Did the authors try to compare their loss function with the one obtained from first order optimality conditions (https://arxiv.org/abs/2210.12153)? Can the authors please provide results of such comparison? \n\n2. **a posteriori guarantees**\n\n   Authors used work \"a posteriori guarantees\" which usually means that we can ensure a certain level of error. For example, in a form of inequality that bound error from above based on the quantities that are computable without the use of exact solutions. On line 147 authors appeal to the law of large numbers to derive their error estimation.\n\n   a. Can the authors please elaborate how one can use the law of large numbers to obtain an upper bound on error?\n\n   b. Should not authors require some properties of probability distribution for the law of large numbers to hold?\n\n3. **weak benchmarks**\n\n   The first batch of examples is given in Table 1. In all cases functions are completely separable, i.e., both function and its conjugate are sums of one dimensional functions. This is also true for the chosen Hamilton–Jacobi equation considered in Section 5.3. The only non-separable function used to test the method is (5.1). It is clear that high-dimensional separable functions are easier to approximate (e.g., hyperbolic cross). Can the authors increase the number of non-trivial benchmarks?\n\n4. **misc**\n\n   a. How $L_2$ errors reported in Table 1 are computed?\n\n   b. How $L_2$ errors reported in Table 3 are computed? \n\n   c. The table on the right panel of Figure 2 contains the claim that authors computed $L_2$ which is not true since convex conjugate is not available.\n\n   d. How direct learning of high-dimensional functions is implemented?\n\n   e. Lines (204-205) \" The ResNet consistently provides the best approximation results despite not guaranteeing convexity ... \" Did the authors try to quantify convexity of learned functions?\n\n   f. Line 78 \"... resulting approximation of $f^{*}$ is automatically convex.\" Potentially misleading since convexity only holds for ICNN not the ResNet.\n\n   g. Equation between lines 66 and 67 right before equation (1.4): right bracket is missing.\n\n   h. Line 47, 48 \"A different approach, going back to a tweet by Peyré [2020]\" and in the footnote \"see e.g., Nesterov (2005), Beck and Teboulle (2012), Niculae and Blondel (2017) or Mensch and Blondel (2018)\" So is it from 2020 tweet or from 2005 Nesterov book (or article)? Why these references are given in plain text?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new deep learning method for computing the Legendre transform of convex and differentiable functions. In particular, the method is based on an implicit form that allows computing the Legendre transform on the codomain of the gradient of the original function. This formulation allows an efficient machine learning formulation based only on a set of samples in the original domain of the function.\nAuthors also address the problem that the measure induced by the gradient of the original function $f$ might not be well distributed in its image, leading to biased datapoints when learning the Legendre Transform; they discuss how to learn an inverse function of the gradient map and use it to obtain well-distributed samples in the domain.\nFinally, authors present numerical experiments of their method with many different network architectures, such as KANs, ResNet, MLPs, and ICNN.\n\nSTRENGTHS AND WEAKNESSES:\nThe method is very straightforward and, most importantly works well, on the examples provided \n- I feel the paper would improve with additional motivation on why numerical Legendre transform is beneficial, maybe with a good numerical example where this method excel. The examples provided are not too impressive.\n- The numerical methods _Deep Legendre Transform_ is compared with are for sure less performant in high-dimension, but are more general: they do not require the function to be convex and differentiable. Moreover in the numerical experiment we have a pure time comparison that can be misleading: DLT requires the computation of the gradient that can hard to compute; in situation where the $f$ (and its gradient) are not available analytically, they could account to the most expensive part of the compute.\n\nQUESTIONS:\n- I do not understand the purpose of Section 3: it does not show and guarantee convergence rate, just exemplification of the law of large numbers and change of variable of an integral. Is something missing or in the Supplementary material?\n- Two questions related to the sampling from a distribution in the gradient space?\n\t- Could adding a penalty if $h_\\vartheta$ outputs outside of $C$, the domain of $f$, be beneficial to the learning instead of discarding those samples?\n\t- Most importantly, I do not get why, if you have learned a representation of the inverse of the gradient $h_\\vartheta$ , you can't just use Eq. (1.2) to directly compute the Legendre Transform, instead of computing it through DLT.\n- Could you give an example where $D$ is a proper subset of the domain of $f^*$ ? In that case, how does the DLT behave outside of its training samples?\n\n**Minors**\n- You use the acronymous ICNN before you specify what it is (line 64)\n- Line 77: why are ResNet convex?\n- Line 164: Why specifically two? Does not depend on the complexity of the function?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}