A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers
Authors: Roman Tarasov
Petr Mokrov
Milena Gazdieva
Evgeny Burnaev
Alexander Korotin
Venue: ICLR 2026 Poster (2026)
Keywords: optimal transport
semi-dual optimal transport
statistical learning theory
approximation bounds
Primary area: generative models
TL;DR: Statistical generalization bounds for (semi-dual) quadratic Neural Optimal Transport solvers

Abstract: Neural network-based optimal transport (OT) is a recent and fruitful direction in the generative modeling community. It finds its applications in various fields such as domain translation, image super-resolution, computational biology and others. Among the existing OT approaches, of considerable interest are adversarial minimax solvers based on semi-dual formulations of OT problems. While promising, these methods lack theoretical investigation from a statistical learning perspective. Our work fills this gap by establishing upper bounds on the generalization error of an approximate OT map recovered by the minimax quadratic OT solver. Importantly, the bounds we derive depend solely on some standard statistical and mathematical properties of the considered functional classes (neural nets). While our analysis focuses on the quadratic OT, we believe that similar bounds could be derived for general OT case, paving the promising direction for future research. Our experimental illustrations are available online https://github.com/milenagazdieva/StatOT.

Published as a conference paper at ICLR 2026
A STATISTICAL LEARNING PERSPECTIVE ON SEMI-DUAL ADVERSARIAL NEURAL OPTIMAL TRANSPORT SOLVERS
Roman Tarasov', Petr Mokrov', Milena Gazdieva 1.2, Evgeny Burnaev Aleexandrr Korotin 1.2 Appllied AT Institute, Moscow, Russia
2 AXXX, Moscow, Russia a rasov..... rbeyandex.ru
ABSTRACT
Neural network-based optimal transport (OT) is a recent and fruitful direction in the generative modeling community. It finds its applications in various fields such as domain translation, image super-resolution, computational biology and others. Among the existing OT approaches, of considerable interest are adver- sarial minimax solvers based on semi-dual formulations of OT problems. While promising, these methods lack theoretical investigation from statistical learning perspective. Our work fills this gap by establishing upper bounds on the general- ization error of an approximate OT map recovered by the minimax quadratic OT solver. Importantly, the bounds we derive depend solely on some stan- dard statistical and mathematical properties of the considered functional classes (neural nets). While our analysis focuses on the quadratic OT, we believe that similar bounds could be derived for general OT case. paving the promising di- rection for future research. Our experimental illustrations are available online https:: /ttess///ittiie wtw.tttt..oooooeeegaaaaaaeeeeeee
INTRODUCTION
In recent years, there has been boom in the development of computational Optimal Transport (OT) which has been identified as a powerful tool of solving various machine learning problems, e.g.,
biological data transfer (Bunne al. 2023 Koshizuka & Sato 2022 Vargas et al. 2022), image
generation (Wang et al.) 2021 De Bortoli et al.) 2021; [Chen et al. (021) and domain translation (Xie et al. 2019 Fan et 2023) tasks. The first works in the OT field had proposed methods for solving OT problems between discrete distributions (Cuturi 2013 Peyré al. 2019). The next milestone was the emergence of OT-based neural methods using the OT cost as loss function for updating the generator in generative models (Sanjabi al.) 2018 Guirajani et al. 2017 Petzka et al.) 2018 Liu et al.) 2019). Only recently, the ML community has experienced an explosion of interest developing scalable neural methods which compute OT map (or plan) and use it directly as the generative model, see, c.g., (Rout et al., 22022] Daniels et al. (021). Methods of this kind are usually attributed as continuous OT solvers and typically have minimax optimization objectives based on semi-dual formulations of OT problems.
Although a large number of neural network based minimax OT solvers have already been developed,
e.g., (Fan et al. 2023 Korotin et al. 2023b at Nguyen et al.) 2024 Buzun et al. 2025 Gazdieva et al.
2023). there is relatively limited understanding of the theoretical aspects of their losses. Existing
theoretical investigations (Makkuva et al. 2020 Rout et al.) 2022 Fan et al. 2023 Asadulaev et al.
2024 Kolesov et al. (2024)) do not reveal practical convergence guarantees, i.e., statistical rates, of these solvers. In contrast, several works non-minimax OT solvers (Gunsilius) 2022 Hütter & Rigollet| 2021) develop statistical bounds for their solutions. However, the analysis of minimax solvers is significantly more complex as their optimization objective includes two variables, OT map and separate dual variable, while non-minimax approaches just operate with only one variable.
Contributions. In paper, fill the aforementioned gap by conducting thorough theoretical investigation of minimax quadratic OT solvers. First, we show that for the minimax OT solvers, the generalization error, i.e., the difference between true and approximate OT maps, can be upper

---

Published as a conference paper at ICLR 2026
bounded by the sum of estimation and approximation errors (Theorem 4.1 Second, specify the bounds for the estimation errors by showing that they depend only on the Rademacher complexities of the classes of the neural nets used (Theorem 4.2) This result highlights the importance of consistent estimation theory in the context of minimax OT solvers. Third, we reveal the approximation ability of OT solvers, i.e., show that the approximation errors can be made arbitrarily small by choosing the appropriate classes of neural nets (Theorems 4.3 4.6) As result, we establish the learnability guarantees for minimax OT solvers by showing that their generalization errors also can be made arbitrarily small by choosing the sufficient number of samples and appropriate classes of neural nets (Theorem 4.9 Corollary 4.10p. We support our theoretical results with an illustrative experiment, $
Notations. In the paper, N.YCR' are convex compact subsets; continuous vellur-valued functions on X' are C()); absolutely continuous prob. distributions on X' are (X). We use the same symbol "p" to denote a continuous distribution p (a)) and its density p(z) at some point x E X'. For measurable map :', we use T to denote the associated push-forward operator. Throughout the text. (1)p())) denotes the L norm w.r.t probability measure p, i.e., for T1, T2: X x
[Ti - - (2 = / [[T)(z) - T2(x)) (2)))))p(())d
For function & C()), its convex conjugate is ; X - R; ((x) def max (2x v) ((v)) (similarly for functions supported on X). Note that convex conjugate is always a convex function.
2 BACKGROUND AND OBJECT OF STUDY
In this section, we provide key concepts of OT theory that are used in our paper, a comprehensive introduction could be found in (Santambrogio 2015 Villani 2008). Throughout the paper, p (X) and q E ()) are absolute continuous source and target distributions.
Monge's OT problem. Given p. q and continuous cost function X y R. Monge's primal OT prescribes to find a measurable transport map T X + y which minimizes OT cost:
Cost, q) def inf [ c(x, ((x))) Tap=y
In other words, we want to find a map T that transports probability mass from p to q in the cheapest way with respect to the given cost function c(x, y). see Fig, [ Le., the average transport expenses when moving to should be minimal.
c(x1,T(x:))
Wasserstein-2 distance. A popular example of an OT cost the (squared) Wasserstein-2 distance (W3), for which the cost function is quadratic, i.e., c(x. y) l| y1lž. In this case, the corresponding OT objective is known permit the unique minimizer 7
X
T(x:)
p X
c(xx.((x))
T(x2) q
T(x.)
c(x,T(xx.)
Semi-dual OT problem for quadratic cost. Primal OT = y/1 has
Figure 1: Monge's formulation of optimal transport.
problem (T) with quadratic cost c(a, y) 1 tight dual counterpar("
(Santambrogio
2015 Thm. 1.39):
| 1
[lvll (y)ddy min PEC())
{ {f ((())(o)dd + [sowaaioaa
Cost 21 ()
1xx 2
p(x)dx++
/
2
where T is the convex conjugate of a potential y E EC()). Potential o* which delivers maximum to 22 is called Kantorovich potential. Removing (2) first two terms that do not depend on so, we get an equivalent formulation:
def min / ((x)p(r)dr+ + / (()))1))d) PEC()) x
((1))
(3)
The optimal potential f which solves (3) is known to be convex ((Santambrogio, 2015 Proposition
1.21)). Therefore, we can optimize 3 with respect to the set of the continuous convex potentials Cvx()) C C())):
Strictly speaking, 2 appears as the dual for Kantorovick's 's relaxation of I1. (Santambrogio/ 2015 $1.1). However, 1 still holds (for the quadratic cost, c(z, y) = | - (antambrogio 2015 Thm. 1.33)
2

---

Published as a conference paper at ICLR 2026
/ F(z)p(x)dx + 1 F(v)a(e)dy.
[(p))
min (ECCx(D))
(4)
Furthermore, we can recover the OT map T which solves ( for quadratic (scalar product) cost from optimal dual potential 7 (a: Vip (ar) (Santambrogio 2015 Thm. 1.17).
Continuous OT problem. Analytical solutions for problems like (2) are, general, not known. In real-world scenarios, the measures p, q are typically not available explicitly but only through their empirical samples X = {a'....., NN ~ p. Y 199.....M... desired solution for primal (dual) OT, two setups possible. In the first setup called discrete (Peyré et al. 2019), one aims to establish optimal matching (point-to-per to-inint correspondence) between the empirical distributions p E de,, q 32 Em=r 6ym. In turn, our study focuses on the second continuous setup. Under this setup. the aim to recover some parametric approximation î of OT map 7 based on available training samples from the reference distributions p. q. Compared to discrete OT, continuous OT allows easy estimation of the recovered map (x="e) ) for new (unseen) samples z=e ~ p. see Fig. 2 contrast, discrete OT solvers perform matching between the empirical distributions and, by default, do not provide out-of -aa--pl estimation of the transport map. Naive interpolation techniques (neural network regression on top of the discrete OT solutions) are known to provide unsatisfactory results, see (Gazdieva al.
~ 9. To approximate the
2023 Appendix D), for the illustration of this M et al.) 2024] PPooladian Niles-Weed (022)) propose more advanced interpolation techniques and develop out-of-sample estimators, their applicability in high dimensions requires a separate study.
issue. While several recent papers (Klein et al. 2024 Deb et al., 2021] Manole
Continuous OT solvers for quadratic cost. In recent years, numerous algorithms have been developed solve the contin- uous optimal transport problem for the quadratie cost. The primal problem is chal- lenging to solve due to the difficulty in satisfying the constraint Tgp q. For this reason, semi-dual problem (3) or (4) solved instead and optimal transport map T is recovered from the optimal dual potential. One popular strategy TTghvaei & Jalali 2019) is to consider (4) and parameterize learned as impul convex neural network (ICNN) (Amos et al. 2017). ICNNs are neural networks which place certain restrictions their weights and activations to the convexity w.r.t. the input. Given proper parametrization of dual potential Pe, 0 E θ, one can directly minimize:
X~p
Y q
Xnew
T' (Xnew)
Figure Continuous setup of OT problem.
C(vp)) (o(z)p(z)dr + poo())(v)ddy.
or its minibatch (Monte-Carle estimate) version. Such methods, however, require additional optimiza- tion steps to compute şo. which may be undesirable from the computational efficiency perspectives. To overcome the issue, researchers have explored ways to simplify the task. Several studies propose methods that approximate the conjugate potential using another neural network (Makkuva et al. 2020 Korotin al., (202 Amos 2023). Such methods, however, may suffer from grudient deviation issue (Korotin et al. 2021b $2) and fail to recover OT mappings sufficiently well.
Continuous minimax OT solvers for quadratic cost. Recently, minimax semi-dual approaches have been actively explored, they learn both the dual potential and the primal transport map. This formulation can be derived by applying the interchange theorem (Rockafellar 2006 Theorem 3A):
p(z)p(a)dz = L (aax((r,1) p(v))p(2)d max / ((x)) ([T())))((dd (5) T
vEy
the outer maxy is taken w.r.t. measurable maps T -
Remark. The existence of a map which maximizes the above problem follows from the measurable selection theory. For y C()), the sets arg max,, {xx y) (p())) }, x E X' are nonempty and closed. From (Aubin & Frankowska 2009 Thm. 8.1.3), follows that there exists a measurable map T with values in such sets. This map delivers maximum to (5)

---

Published as a conference paper at ICLR 2026
In light of ( problem
B is thus reformulated minimax optimization problem:
/ def min max C(s, T).
min max T
(6)
T
Under certain assumptions one may guarantee that if the values ((p 7) and L(v", T are close, then T' is close to OT map T". And it is work which establishes practical convergence guarantees for (6) under continuous OT setup with yp, T' given by neural networks, see $4
Importantly, the use of minimax semi-dual OT losses like 6 is emerging direction in generative
modeling (Korotin et al.
bb22bb Rout et al. 2022 Gazdieva et al. 2025: Korotin et al. 2023a b
Fan al. 2023; Gazdieva et al. 2023; Choi et al.
2024a Buzun al., |025]]/ Asadulavv et al.
2024 Nguyen et al. 2024; Carrasco et al. 2023; Kolesov et al., 2024a), making the analysis of (6) highly beneficial and desirable the community. In particular, some of the minimax OT works, e.g., (Korotin et al. 2021b Rout et al. 2022), explore quadratic (scalar product) cost functions, i.e.,
deal exactly with (6). The others consider more general cost functions (Fan et al., (023) and OT formulations (Asadulaev et al. 2022). Extending the results of our study for such setups is a fruitful direction of future research.
In practice, T and yp are parametrized as neural networks Tw., w E Ω and Po, E O. Besides, following our continuous OT setup, the (unknown) reference distributions p. q are replaced by their empirical counterparts p. q. This yields the optimization problem typically solved in practice:
Object of study.
(In (w po (Tn(
M Go (Um)
min max BEB WES
def
min max Lipo: Tw)- BEO
N
M
WEEl
ML practitioners and researchers naturally wonder how different the practical map T.- that solves 7 and OT map T*. Formally speaking, the problem is to estimate the generalization error
E
T - Tu- 12()))
(8)
where the expectation is taken w.r.t. empirical training samples X --,,Y p, q. Remarkably, the presence of the error (8) comes from two sources: a) reliance on the restricted families of optimized potentials, maps (approximation error); b) usage of empirical measures p. q instead of original ones (estimation error). Establishing the particular bounds for (8) is our primal focus, see $4
3 RELATED WORKS
In this section, we discuss the existing works which conduct theoretical analysis of semi-dual OT losses. The current progress in this field could be subdivided into two branches reviewed below.
Non-minimax semi-dual OT. The first branch of works (Hütter & Rigollet 2021 Gunsilius 2022) analyze the non-minimax losses (3) and and develop error bounds (8) for pushforward maps 7 given by the gradient of dual potentials, i.e., E V Voull??.0) To achieve particular statistical
X.Y
rates, the authors place certain restrictions on the considered problem setup and deal with specific classes of functions (maps), e.g., wavelet expansions or (kemne-ssmoothe)) plug-in estimators. Recent studies in this direction (Divol et al. 2025 Ding et al. 2024) extend the analysis of error rates for the class of potentials yp given by neural networks, e.g., input convex neural networks. Mcanwhile, none of the mentioned works treats the learned map T separately from dual potential yp. Importantly, the analysis of minimax objective (6) is considerably more challenging than (3) due to the additional "degree of freedom" given by the optimized map T. Furthermore, saddle-point problems such as (6) are known to more tricky for theoretical investigation than usual minimization. At the same time. the practical demand stemming from the recent proliferation of minimax- makes such an investigation highly desirable. All of this necessitates separate study of minimax OT solvers; the adaptation of existing non- minimax results is questionable, if that is even possible.
OT solvers, see $2)
Minimax semi-dual OT. The estimation of (8) in case of minimax OT 77) is a much less explored task. In fact, there are no studies at all that provide statistical bounds for recovered minimax OT
map T.,. The existing works (Makkuva et al. 2020 Rout et al. 2022) only conduct an analysis of
| - T 1l in the sense of duality gaps. That is, the error between the true OT map and recovered
4

---

Published as a conference paper at ICLR 2026
map is upper-bounded by values of functional [(p, T). The duality gaps analysis helps to validate minimax methodology; it is common for more broad class of (non-quadratic) minimax OT solvers (Fan et al. 2023 Asadulaev et al. 2024 Kolesov et al. 2024a). However, we again emphasize that it does not reveal particular statistical rates and guarantees for (8. In (González-Sanz, et al. 2022), the authors establish promising results for the error of recovered approximate OT map. However, their problem formulation of minimax OT differs from ours, eq. (6). as they aim to treat OT as the limit of regularized GAN objectives. Finally, the work (Nietert & Goldfeld (025) conducts theoretical analysis of the error of solving the minimax Wasserstein distributionally robust optimization problem which again differs from ours.
Theoretical analysis of other OT formulations. For the completeness of exposition, we also mention several studies that develop statistical learning analysis of recovered OT map (plan) error for non semi-dual OT formulations. The works (Genevay et al. 2019, Rigollet & Stromme 2022
González-Sanz & Hundrieser 2023 Mokrov et al., 2024 Goldfeld et al. 2024 Korotin et al. 2024
Kolesov et al. (2024b) deal with Entropy-regularizee OT (uushchin et al. 2023); (Vacher & Vialard
2022; 2023] Gazdieva et al. 2024) investigate unbalanced OT versions; (Benamou & Brenier) 2000)
considers dynamic OT. Although these works are interesting and insightful, their object of study is different from ours, making them less relevant.
Since the minimax objective 66) resembles adversarial training, additionally discuss the literature addressing the theoretical aspects of Generative Adversarial Neural Networks (GAN) losses and their relation to semi-dual OT losses in Appendix [ For comprehensive analysis of the connections and differences between these losses, we refer the reader to (Gazdieva et al. 2025).
4 THEORETICAL RESULTS
In real-world use cases, OT practitioners are given (i.id.) training samples X~ ~ and Y ~~ q and optimize empirical minimax objective ( with respect restricted classes of functions jo E F, E e.g. neural networks. Below we denote the solutions of the problem which we have in practice (superscript "R" stands for "Restricted"):
GRt argmin max Lige, T); PEF TET
7 argmax E (G" ,T)
TET
Note that in these equations we implicitly assume the existence of optimizers @R E F, F, ET. While general this may not always hold true, some natural practical choices of F and T, e.g.. neural network architectures yo, Tw with bounded set of parameters O Ω guarantee the existence.
Our goal is to estimate the generalization error, i.e., the (average) error between OT map T and empirical map T..
E x E - T -
LP(p))
We subdivide the problem into several steps. First, we upper-bound the error using differences in the functional C(p, T) values 1$4.1). The obtained upper bound decomposes into several terms:
estimation and approximation errors that occur in both the inner and outer optimization problems within our minimax objective (6) Second, we estimate each term individually using suitable techniques from statistical learning theory ($4.2). Finally, bring all together and formulate main theoretical result ($4.3). We provide the proofs of all thcoretical results in Appendix A
4.1 ERROR DECOMPOSITION
We start with introducing the components which will appear when upper- 9 The primary quantity which we analyze in this section is the error between values of functional Z, i.c.:
L(p, T) L(y',TT).
(10)
Depending on the context, the plug-in arguments yp, T, the expression above may be "optimal" in some sense and connected to each other. In particular, the potentials yp, yp' may solve the outer (min,p) optimization for the original (6) or empirical (7) objective in certain class of functions, i.e., C()) F. In turn, the maps T, T" can be optimal in the inner (maxy) optimization of the minimax objectives for certain potentials oo and also belong to certain classes of functions. These choices give rise to several options, which are discussed below.
5

---

Published as a conference paper at ICLR 2026
Typically, quantities (errors, bounds, etc.) appearing in statistical learning theory allow for decompo- sition into approximation and estimation components. We also treat error (10) in this framework.
Approximation error. This error arises because are not optimizing over the full space of continuous functions E C()) and measurable maps T, but over the restricted classes F and of neural networks. Since neural networks have limited capacity, they may not be able to represent the true saddle-point solution (e²", , T*) of ((, T) exactly. We consider two components of this error, focusing respectively on the outer and inner optimization in (6).
The outer component assumes that the inner maximization (maxT) is solved exactly. We want to identify which extent the restricted outer minimization (minger) affects the minimax objective:
EA (F)
del
min max C(p, T) min max L(p,
PEf T
Note that maxy C(p,, [(p)) Then the outer approximation error could be reformulated as:
Edws (F) =
min
[(p) min [(p)
(11)
pef
The inner approximation emor, in turn, measures the looseness of inner maximization in the restricted class T, i.e., the gap maxy L(s,, T) - maxTeT L({,T)) To have ye-independent characteristic, we further bound the gap uniformly w.r.t. yp E F. This yields
(F T) max WEF
max C(p, max C(p, TT)
(12)
TET
Estimation error. practice, we do not have direct access to L(p, T) but instead optimize the empirical functional Elgp T) based on finite training data. Thus, the optimal potential and map which deliver optimal value to Z. i.e., . and PR may be different from those which deliver optimal value to f., even within classes F, This introduces estimation error, which quantitatively measures the behaviour of functional L which takes "empirically-optimal functions & T as the input. By design, the estimation error is evaluated within classes F, T. Also, since G I are (indirectly) based on the empirical samples {x1..... .N X p and ,.I,...,MM. error is averaged w.r.t. them and depend on the sample sizes N, M.
Y q, the estimation
The inner estimation error is defined as:
E (F,
def
E X.Y
max L(G
SeRR,T T) L("", TT
(13)
TET
For the fixed potential & it shows how different are the optimal value of the inner (maxre7) optimization and the "empirically-indue -induceed" one L(GR (R..
Similarly, the outer estimation error deals with the outer minimization (min while assuming exact solution of inner maximization in class T:
Ea (FT.N,,)
= max C("", min max ((p, T)). x.Y TE7
(14)
PEF TET
Now we are ready to formulate our important Theorem 4.1 about the error decomposition.
Theorem 4.1 (Error decomposition). Let F be class of B-strongly convex convexfunctions functions, then
4 (x(p) S (E M) 3 (F, () EE -
E X.Y
(F)) (15)
The theorem shows that the (averaged) 2 gap between the empirical and true OT maps 9 could be upper-bounded by our introduced errors. Importantly, the latter compares the values of functional L and thus easier for the analysis (conducted in the next subsection $4.2) than the former.
4.2 BOUNDS ON THE APPROXIMATION AND ESTIMATION ERRORS
In this section, we establish the bounds the estimation and approximation emrors of the minimax OT solvers defined in 1 In the theorem below, we establish the bounds on the total estimation error, ie., the of inner (13) and outer emrors (4). The bounds given by Rademacher complexities of the certain classes of functions. Rademacher complexity is a standard tool statistical learning theory; recall its definition in Appendix A.2

---

Published as a conference paper at ICLR 2026
Theorem 4.2 (Rademacher Bound the Estimation Error), Let E = Ef + E.Su. Then E S 8R (H) + (F),
(16)
where H(F,T) def {h h(x) - (a, (a, T(x)) - ((((2)), E T.P E F} and Rp.N (H) is the Rademacher complexity of the function class H w.r.f. probability density p for sample size N.
Now we proceed to the approximation errors 112 and (II). In what follows, we prove that under proper choice of classes F, T the enrors could be done arbitrarily small.
First we look at the inner approximation error (12. Note that depends on both classes F and T. The following theorem states that by imposing certain restrictions class F and by choosing an appropriate class of neural networks T. we can control the inner approximation error.
Theorem 4.3 (Inner approximation error). Let F C C(V) be class of Lipschitz 3-strongly convex functions, totally bounded with respect to the Lipschitz norm. Then for all e >> 0 there exists a class of neural networks Tub TT F) and their Im-projected versions T T= (projy oT, T Tub) s.L.:
(F, <s;
(17)
projy: RD + is the projection operator, see Lemma A.7 in Appendix A.3 for the formal definition.
Importantly, the projection operator is not a restrictive add-on; it is usually used in practice, for example when working in the image domain (projection to pixel space). Our following Proposition 4.4 elaborattss on the functional class F appearing in the theorem above.
Proposition 4.4 (Class F in practice). Let Fienn be a class of K-layer ICNN with CELU activations. bounded width, height; bounded weights matrices [|all2 S k= K; defined in proof of (Chen et al. 2018 Thm. 1). Then F=(++ 3 |1 &p E satisfies the requirements of Theorem 4.3 i.e. is class of B-strongly convex functions totally bounded w.r.t the Lipschitz norm.
From the proof of Theorem 4.3 one can notice that the class of neural networks T for which 17 holds could be chosen be finite. This naturally leads to another practically Remark 4.5
Remark 4.5, In conditions of Theorem 4.3 the class of maps T could be chosen be a ReLU Neural Network (MLP) with fixed width, height; with weights of bounded norms, Explicit capacities (e.g..
Rademacher complexities, covering numbers, VC dimensions) of neural nets' classes are well-studied
in the literature (Golowich et al. 2018 Ou & Bölcskei 2024 Bartlett et al. 2019 Bach 2017).
Now we turn to the analysis of the outer approximation error (TT). Similarly to the inner case, under the assumption that the optimal potential y- f3-strongly convex for some B> O, we show that there exists Neural Network architecture for the dual potential yo which makes the error arbitrarily small.
Theorem 4.6 (Outer Approximation Error). Let the optimal potential yp" be B - ssrongly convex. For arbitrary& > 0 there exists totally bounded w.r.t. Lipschitz norm class F = F(B,E) (does not depend on xp" such that for some B-strongly convex and Lipschitz.function Lt holds that vft E F and ((v2)) - L(p²)) <.
Corollary 4.7. There exists class of Lipschitz, B-stroogly convexffunction totally bounded with respect to the Lipschitz norm, such that
Eu (F) Sc.
(18)
In our Theorem 4.6 (and Corollary 4.7p we assume that the optimal potential 4o" is B - strongly convex. Our Remark clariffis the conditions on the source and target distributions p and q which ensure the strong convexity.
Remark 4.8. Recall that the support subsets X' 3 be convex; let probability densities and be strictly positive and bounded on X and V, respectively; additionally, let probability densities p. q be Hölder continuous functions, i.e., p E C (X)); q E C ()") for some a > 0. Then, from Caffarelli 1996), Theorem ((C3), it follows that optimal potential fp is 3-strongly convex for some O.
is noteworthy that the conditions on and g are rather flexible; they are usually met for distributions that commonly used in practice (with bounded densities) and for mixtures of distributions, for example mixtures of Gaussians (with the support artificially limited to a large enough ball (o) R. > 1, ensure the compactness of X, Y))
7

---

Published as a conference paper at ICLR 2026
4.3 BOUND ON THE GENERALIZATION ERROR
The main goal of our paper is the establishment of bounds on generalization error, i.e., the difference in true OT map and its empirical approximation defined in (9). In the theorem below, the previously obtained bounds on the estimation (Theorem
4.2 and approximation emrors (Theorems
4.6 to derive bound on the generalization error.
Theorem 4.9 (Bound the Generalization Error). Let the optimal dual potential sp be B-strongly TE,, that convex. Then for any E > 0 there exist such classes F = F), =
N IT P (260) S
32 B (Rp.x(H) + (x))
(19)
Moreover, and could be chosen as neural networks following Proposition 4.4 and Remark [.3
Theorem 4.9 shows that a practitioner can make the generalization error g arbitrarily small by choosing appropriate classes of functions F and T. However, the numerical convergence rates remain unclear. The next corollary shows for the particular choice of functional classes (neural network), the obtained bound can be specified by replacing the Rademacher complexities with their upper bounds depending only on the number of empirical samples (N, MT).
Corollary 4.10 (Generalization Error for the Specific Classes of Neural Networks). For any 8 > let T, F be classes of neural nets specified in Theorem 4.9 Then inequality (19) turns to
E
T - Lalp)
+
(20)
The corollary allows us to conclude that the generalization error can be made arbitrarily small if one selects the appropriate classes of neural nets and sufficient number of empirical training samples.
5 EXPERIMENTAL ILLUSTRATIONS
In this section, we experimentally illustrate how the generalization error behaves in practice. Specif- ically, we show that the empirically observed generalization errors, i.e., the convergence rates of OT solver, are close to the bounds which we theoretically derive in our Corollary 4.10 For clar- ity of illustration, we study these rates for the examples where the generalization error is in the proximity of either the estimation error ($5.1) or the approximation error ($5.2). For both exper- iments, we utilize the benchmark from (Korotin et al., 2021b) which provides ground- -ruth OT maps T and potentials ip" for the given source and target measures. In fact, the benchmark data constructed based on the potential 6* parametrized by the specific NN architecture (ICNN). We consider their Gaussian mixture experiment, where the first distribution corresponds to mix- ture of three Gaussians and the second to an approximate mixture of ten Gaussians. We refer to (Korotin al. 20221] Appendix A) for additional details on the experimental setup. Technical training details are given in Appendix D The source code is written in PyTorch and is available online httts:://ggtttat
com/ letagaziieva//tattot
5.1 EXPERIMENT ON THE ESTIMATION ERROR
Below we study the effects of the estimation error coming from the limited number of data samples available for training. For this purpose, use the same neural network architecture for
loquo of ieneeaiizaaionnee error
DIM=2
DIM=4
Experimente Slope
0.0
-0.5
depoo
90000000
-1.0
Sopen -0722
potential which was used for the ground-truth potential op' in the benchmark. allows us to assume that the outer approximation error (TT) arbitrarily small. Then, for fixed parametriza- tion of OT maps Ta, the generalization error (99 reduces estimation error (up to additive constant).
-2.0
-2.5
-3.0
Slooc
Experimental arer
logi: N.M
Ingio N,M
Figure 3: Convergence rates of the OT solver learned with the quadratic transport cost and limited number of empirical training samples.
Details. We consider distributions of dimension D-2, 4 and OT solver with the quadratic cost c(z, y)= po0=|a--00 .l... To show the dependence of the convergence rates the number of available empirical samples N,M, M, randomly select different numbers of samples (from 1 to 104) from the source and target datasets, and train the OT solver using these samples. We set ICNN as the potential you and an MLP with ReLU activations
8

---

Published as a conference paper at ICLR 2026
as the transport map Tw. During validation, calculate the differences between the learned T and for all choices of N, M and report them in Fig. We plot the error using the logarithmic scale. Additionally, we rum a linear regression on the logarithms of the error, as well as the logarithms of N. M. Then we plot the resulting line, which describes the relationship between these variables in log1o-log10 space.
true 7 OT map ||T 7 1 (::
Results show that the logarithm of the error exhibits approximately linear convergence w.r.t. the logarithms of N, M (with slope Σ -0.5). supports our theoretically derived bounds on the error, Corollary 4.10 Still, we note that the theoretical bounds may not hold in more complex practical setups due to the optimization errors coming from particular optimization procedures which are typically used to train the networks. In Appendix C.I we report the estimation errors of several trivial baselines. Estimation errors the OT solver for higher dimensions are analyzed in Appendix
5.2 EXPERIMENT ON THE APPROXIMATION ERROR
In this section, we experimentally illustrate the effects of the approximation error coming from the limited expressivity of the architectures of neural nets Tw and pg. For this pur- pose, we follow the experimental setup described in $5.T with two important differences. First, we take ~ 10M data samples from the benchmark, which allows us to assume that the estimation error is rather small and, thus, the generalization error (9 reduces the approximation one. Second, we consider more shallow architectures for potential than the one proposed in the benchmark, which ensures that the approximation error does not vanish.
Details. We consider neural nets ye of the same depth in the architecture for the ground-truth potential s", but with a limited size of hidden layers Hạ (max He from to 64). Note that for sp" this value is equal to 64. Besides, we vary the width of the NN architecture for the transport map T. We consider the architecture of the same depth as in our previous experiments, but with varying size of hidden layers max Hy (from
DIM=2
DIM=4
logio of approx. error
0.5
logso approx. error
0.55
-1.0
max't
maxi
-1.5
maxh, -
16
maxit, 16
maxh, 32
maxii, -2
-2.0-
mavit
maxit,
1 to 8). In our previous experiments, we set max l/7 = 512. For each experiment, we calcu- late the difference between the learned and true OT maps: ||T - T" (12=ppp) as explained above. We plot the results using the logarithmic scale in Fig.
maxH, Figure 4: Empirical approximation error of the OT solver learned with the quadratic transport cost and using shallow NN architectures.
maxH-
Results show the expected behavior with the increase of the nets' width, the approximation error becomes smaller; when the potential architecture coincides with the one which was used during the benchmark construction (max H. 64), the approximation error is close to zero. Still, the trend might be corrupted by the optimization error, leading to outliers in the particular error values. Besides, the reader might notice that for nets' architectures with extremely small width, the approximation error might collapse to particular large values. Note that this is an expected behavior since the nets with too shallow architectures have very limited expressivity which might lead to improper solutions and large approximation errors. For clarity, we visualize the solver's solutions in Appendix C.3
DISCUSSION
Our paper performs theoretical investigation of semi-dual minimax OT solvers, i.e., popular and fruitful branch of generative models based on Optimal Transport. While these solvers show impressive results in various machine learning problems, theoretical investigation of their practical convergence guarantees is rather limited. We address this problem by presenting the first theoretical analysis of these minimax solvers from statistical learning perspective. Strietly speaking, some of the established results are more aligned with the consistent estimation theory framework rather than statistical learning one. Most importantly, our paper provides learnability guarantees for the solvers which justify their practical usability. Apart from the theoretical results, our paper provides experimental illustrations supporting the theoretical conclusions. We believe that our research will advance the development of minimax adversarial OT solvers by mitigating potential concerns regarding their theoretical validity.

---

Published as a conference paper at ICLR 2026
Limitations. One limitation of our study corresponds to the focus popular case of quadratic OT cost function. Besides, one can notice that our analysis employs quite strict B-strong convexity assumptions and does not provide lower bounds the generalization error. Additionally, do not provide practical guidance on selecting hyperparameters of neural nets parametrizing the map and potential in OT solvers. We discuss the validity of the assumptions, lower bounds and practical relevance of our analysis in Appendix B Generalization of the established bounds for the general OT formulations represents promising avenue for future work.
Reproducibility. A full list of the experimental details is provided in Appendix D Our experiment uses synthetic dataset (benchmark) which is constructed via publicly available code of (Korotin et al.) 2021b). We provide the code for the experiments in supplementary material.
7
ACKNOWLEDGMENTS
The work was supported by the grant for research centers in the field of Al provided by the Ministry of Economic Development of the Russian Federation in accordance with the agreement W00000C313995PPF0000 and the agreement No 139-10-2025-03.
REFERENCES
Brandon Amos. On amortizing convex conjugates for optimal transport. In The Eleventh International Conference on Learning Representations, 2023. URL wwt.hhttsss/ooto /ttps:://ppereeieew hwtp:/wwwwww..eeeeee....e...ee wwt. formm?
ad=TQ55UwS 4ai
Brandon Amos, Lei Xu, and Zico Kolter. Input convex neural networks. In International conference on machine learning, pp. 146-155. PMLR, 2017.
Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (gans). In International conference machine learning, pp. 224-232. PMLR, 2017.
Arip Asadulaev, Alexander Korotin, Vage Egiazarian, Petr Mokrov, and Evgeny Burnaev. Neural opti- mal transport with general cost functionals. In The Twelfth International Conference on Learning Representations, 2024. URL/https:/ /tt.tttpss//peereviieeeeee
net / fow.forrmmimo /tt://ooommimi1111111-1-1-1
Jean Pierre Aubin and Helene Frankowska. Set-valued analysis. Modern Birkhauser classics, 2009.
Francis Bach. Breaking the curse of dimensionality with convex neural networks. Journal of Machine Learning Research, 18(19):1-53, 2017.
Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mehrabian. Nearly-tight ve-dimension and pscudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20(63):1-17, 2019 2019.
Jean-David Benamou and Yann Brenier. A computational fluid mechanics solution to the monge- kantorovich mass transfer problem. Numerische Mathematik, 84(3):375-393, 2000.
Gérard Biau, Benoît Cadre, Maxime Sangnier, and Ugo Tanielian. Some theoretical properties of gans. The Annals of Statistics, 48(3):153911566 2020.
Gérard Biau, Maxime Sangnier, and Ugo Tanielian. Some theoretical insights into wasserstein gans. Journal of Machine Learning Research, 22(119): 2021.
Charlotte Bunne, Stefan Stark, Gabriele Gut, Jacobo Sarabia Del Castillo, Mitch Levesque, Kjong-Van Van Lehmann, Lucas Pelkmans, Andreas Krause, and Gunnar Rätsch. Learning single-cell perturbation responses using neural optimal transport. Nature methods, 20(11):1759-7768 2023.
Nazar Buzun, Maksim Bobrin, and Dmitry V Dylov. Expectile regularization for fast and accurate training of neural optimal transport. Advances Neural Information Processing Systems, 37: 11981 1-119837, 2025.
10

---

Published as a conference paper at ICLR 2026
Luis A Caffarelli. Boundary regularity of maps with convex potentials -i. Annals of mathematics, 144(3):453-44 4963,49966.00 1996.
Xavier Aramayo Carrasco, Maksim Nekrashevich, Petr Mokrov, Evgeny Burnaev, and Alexander Korotin. Uncovering challenges of solving the continuous gromov-wasserstiin problem. arXiv preprint arXiv:2303.05978. 2023.
Anish Chakrabarty and Swagatam Das. On translation and reconstruction guarantees of the cycle- consistent generative adversarial networks. Advances in Neural Information Processing Systems, 35:23607-23620, 2022.
Tianrong Chen, Guan-Homg Liu, and Evangelos Theodorou. Likelihood training of schrödinger bridge using forward-backwang sdes theory. In International Conference on Learning Representations, 2021.
Yize Chen, Yuanyuan Shi, and Baosen Zhang. Optimal control via neural networks: A convex approach. arXiv preprint arXiv: 805. 11835, 2018.
Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang, Generative modeling through the semi- formulation of unbalanced optimal transport. Advances in Neural Information Processing Systems, 36, 2024a.
Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Analyzing and improving traimal-t based adversarial networks. The Twelfth International Conference on Learning Representations, 2024b. URL
wwwp:/.ee/eeeeeeeee / id- jODehvt
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Max Daniels, Tyler Maunu, and Paul Hand. Score-based generative neural networks for large-scale optimal transport. Advances in neural information processing systems, 34:1 129511229655, 2021.
Valentin De Bortoli, James Thortoo,, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications to score-based generative modeling. Advances in Neural Information Processing Systems, 14:17695-17709, 2021.
Nabarun Deb, Promit Ghosal, and Bodhisattva Sen. Rates of estimation of optimal transport maps using plug-in estimators via barycentric projections. Advances in Neural Information Processing Systems, 114:29336-2955.3 2021.
Pinar Demetei, Rebecca Santorella, Björn Sandstede, William Stafford Noble, and Ritambhara Singh. Scot: single-cell multi-omics alignment with optimal transport. Journal of computational biology. 29(1):3-18, 2022.
Yizhe Ding, Runze Li, and Lingzhou Xue. Statistical convergence rates of optimal transport map estimation between general distributions. arXiv preprint arXiv:2412.0806/ 2024.
Vincent Divol, Jonathan Niles Weed, and Aram-Alexante Pooladian. Optimal transport map estimation in general function spaces. The Annals of Statistics, 53(3):963-988. 2025.
Owen M Dugan, Peter Y Lu, Rumen Dangovski, Di Luo, and Marin Soljaeic. Q-flow: genera- tive modeling for differential equations of open quantum dynamics with normalizing flows. In International Conference on Machine Learning, pp. 1879-8901. PMLR, 2023.
Jiaojiao Fan, Shu Liu, Shaojun Ma, Hao-Min Zhou, and Yongxin Chen. Neural monge map estimation and its applications. Transactions on Machine Learning Research, 2023, ISSN 2835-8856. URL https:://ppenreviet net / 200t Featured Certification.
Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Extremal domain translation with neural optimal transport. Advances in Neural Information Processing Systems, 36:40381-40413, 2023.
Milena Gazdieva, Arip Asadulaev, Evgeny Burnaev, and Alexander Korotin. Light unbalanced optimal transport. In The eightt Annual Conference on Neural Information Processing Systems, 2024.

---

Published as a conference paper at ICLR 2026
Milena Gazdieva, Petr Mokrov, Litu Rout, Alexander Korotin, Andrey Kravchenko, Alexander Filip- pov, and Evgeny Burmaev. An optimal transport perspective on unpaired image super-resolution Journal of Optimization Theory and Applications, 207(2):40, 2025.
Aude Genevay, Lénaic Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics, pp. 1574-1583. PMLR, 2019.
Ziv Goldfeld, Kengo Kato, Gabriel Rioux, and Ritwik Sadhu. Limit theorems for entropic optimal transport maps and sinkhorn divergence. Electronic Journal of Statistics, (8(1):980-104) 2024.
Noah Golowich, Alexander Rakhlin, and Ohad Shamir. Size-independens sample complexity of neural networks. In Conference On Learning Theory, pp. 297-299. PMLR, 2018.
Alberto González-Sana and Shayan Hundrieser. Weak limits for empirical entropic optimal transport: Beyond smooth costs. arXiv preprint M.XXv:2330.097455. 2023.
Alberto González-Sanz, Lucas De Lara, Louis Béthune, and lipschitz optimal transport maps. arXiv preprint arXiv:2202007665, 2022.
Loubes. Gan estimation of
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
Florian F Gunsilius. On the convergence rate of potentials of brenier maps. Econometric Theory, 38 (2):381-417. 2022.
Nikita Gushchin, Alexander Kolesov, Petr Mokrov, Polina Karpikova, Andrei Spiridonov, Evgeny Burnaev, and Alexander Korotin. Building the bridge of schrödinger: A continuous entropic optimal transport benchmark. Advances in Neural Information Processing Systems, 36:18932- 18963, 2023.
Chin-Wei Huang, Ricky T. Q. Chen, Christos Tsirigotis, and Aaron Courville. Convex potential flows: Universal probability distributions with optimal transport and convex optimization. In International Conference on Learning Representations, 2021. URL htttps://peereviii peet/forum?iic-teePPHHsss.
Jan-Christian Hütter and Philippe Rigollet. Minimax estimation of smooth optimal transport maps. The Annals of Statistics, 4962):1166-1194, 2021.
Kaiyi Ji and Yingbin Liang. Minimax estimation of neural net distance. Advances in Neural Information Processing Systems, 2018.
Kaiyi Ji, Yi Zhou, and Yingbin Liang. Understanding estimation and generalization enror of generative
adversarial networks. IEEE Transactions on Information Theory, 67(5):314-31129, 2021.
Sham Kakade, Shai Shalev Shwaar-SSwartz, Ambuj Tewari, et al. On the duality of strong convexity and Learning applications and matrix regularization. Unpublished Manuscript, http://ttic. uchicago.
strong smoothness:
KaaaeeeSseaaaaaahaeeeeeeee Tewari099 pdf, 2(1):35, 2009.
Patrick Kidger and Terry Lyons. Universal approximation with deep narrow networks. Conference on learning theory, pp. 2306-2327. PMLR, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014
Dominik Klein, Théo Uscidda, Fabian Theis, and Marco Cuturi. Genot: Entropic (gromov) wasser- stein flow matching with applications to single-cell genomics. Advances in Neural Information Processing Systems, 37:103897-1009444, 2024.
12

---

Published as a conference paper at ICLR 2026
Alexander Kolesov, Petr Mokrov, Igor Udovichenko, Milena Gazdieva, Gudmund Pammer, Evgeny Burnaev, and Alexander Korotin. Estimating barycenters distributions with neural optimal transport. In Forty-first International Conference on Machine Learning, 2024a.
Alexander Kolesov, Petr Mokrov, Igor Udovichenko, Milena Gazdieva, Gudmund Pammer, Anastasis Kratsios, Evgeny Burnaev, and Aleksandr Korotin. Energy-guided continuous entropic barycenter estimation for general costs. Advances in Neural Information Processing Systems, 17:107513- 107546, 2024b.
Nikita Kornilov, Petr Mokrov, Alexander Gasnikov, and Aleksandr Korotin. Optimal flow matching: Learning straight trajectories in just one step. Advances in Neural Information Processing Systems, 37:104180-104200 2024.
Alexander Korotin, Vage Egiazarian, Arip Asadulaev, Alexander Safnn, and Evgeny Burnaev. Wasserstein-2 generative networks. In International Conference on Learning Representations, 2021a.
net/ wtt.:eeee//oooooddeooooooeo
EXsa
Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, Alexander Filippov, and Evgeny Burnaev. Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark. Advances in neural information processing systems,
2021b.
Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Kernel neural optimal transport.
In The Eleventh International Conference on Learning Representations, 2023a. URL https
//openrevies, wwt.eee..eeeeeeeeeeeeeeeeeeeee
www.nnet.foruu fttt//oorm?iiddz1/ d
Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Neural optimal transport. In
The Eleventh International Conference on Learning Representations, 2023b. URL https::/
openreview net/ oreareerieev.ntt//ooomr//aaaaaaa (eteet/oooooddddddBBBBA-
Alexander Korotin, Nikita Gushchin, and Evgeny Burnaev. Light schrödinger bridge. The Twelfth International Conference on Learning Representations, 2024. URL https::/
Takeshi Koshizuka and Issei Sato. Neural lagrangian schrödinger bridge: Diffusion modeling for population dynamics. In The Eleventh International Conference on Learning Representations, 2022.
Fabian Latorre, Leello Tadesse Dadi, Paul Rolland, and Volkan Cevher. The effect the intrinsic di- mension the generalization of quadratic classifiers. Advances in Neural Information Processing Systems, 34:21 128-1138211 149, 2021.
Yunwen Lei, Tianbao Yang, Yiming Ying, and Ding-Xuan Zhou Generalization analysis for contrastive representation learning. In International Conference on Machine Learning, pp. 19200- 19227. PMLR, 2023.
Tengyuan Liang. How well can generative adversarial networks learn densities: A nonparametric view. arXiv preprint arXiv:1712.00m 1712.0824, 2017.
Huidong Liu, Xianfeng Gu, and Dimitris Samaras. Wasserstein gan with quadratic transport cost. In Proceedings of the IEEE/CVF international conference on computer vision, pp.
2019,
Shiao Liu, Yunfei Yang. Jian Huang, Yuling Jiao, and Yang Wang. Non-asymptotic error bounds for bidirectional gans. Advances in Neural Information Processing Systems, 34:12328 123399, 2021.
Ashok Makkuva, Amirhossein Taghvaei, Sewoong Oh, and Jason Lee. Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning, pp. 6672-6681. PMLR, 2020.
Tudor Manole, Sivaraman Balakrishnan, Jonathan Niles- Weed, and Larry Wasserman. Plugin estimation of smooth optimal transport maps. The Annals of Statistics, 52(3):966 968-998, 2024.
Andreas Maurer. A vector-contraction inequality for rademacher complexities. In Algorithmic Learning Theory: 27th International Conference, ALT 2016, Bari, Italy, October 19-21, 2016, Proceedings 27, pp. 3-17. Springer, 2016.
13

---

Published as a conference paper at ICLR 2026
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018.
Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin Solomon, and Evgeny Burnaev. Large-scale wasserstein gradient flows. In A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.). Advances in Neural Information Processing Systems, 2021. URL https:// openreview net/ forum? ?id=n11 (ateeennniiusstaSS IuHsMHp
Petr Mokrov, Alexander Korotin, Alexander Kolesov, Nikita Gushchin, and Evgeny Burnaev. Energy- guided entropic neural optimal transport. In The Twelfth International Conference on Learning Representations, 2024. URL
Nikita Moriakov, Jonas Adler, and Jonas Teuwen. Kernel of cyclegan as principle homogeneous space. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, Apr 30 2020. International Conference on Learning Representations, ICLR, 2020.
Bao Nguyen, Binh Nguyen, Hieu Trung Nguyen, and Viet Anh Nguyen. Generative conditional distributions by neural (entropic) optimal transport. In Forty-first International Conference on Machine Learning, 2024.
Sloan Nietert and Ziv Goldfeld. Estimation of stochastic optimal transport maps. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Weigutian Ou and Helmut Bölcskei. Covering numbers for deep relu networks with applications to function approximation and nonparametric regression. arXiv preprint arXiv:2410.06378, 2024.
Zhaoqing Pan, Weijie Yu, Xiaokai Yi, Asifullah Khan, Feng Yuan, and Yuhui Zheng. Recent progress generative adversarial networks (gans): A survey. IEEE access, 7:36322-3665 -6333,, 2019.
Henning Petzka, Asja Fischer, and Denis Lukovnikov. On the regularization of wasserstein gans, In International Conference on Learning Representations, 2018.
Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends@ Machine Learning, 11(5-6)335-607) 2019.
Aram-Alexandre Pooladian and Jonathan arXiv preprint arXiv:2109. 2004, 2021.
Entropic estimation of optimal transport maps.
Philippe Rigollet and Austin Stromme. On the sample complexity of entropic optimal transport. arXiv preprint 22Xvv:220..13472 2022.
R Tyrrlll Rockafellar. Integral functionals, normal integrands and measurable selections. In Nonlinear Operators and the Calculus of Variations: Summer School Held in Bruxelles 8-19 September 1975, pp. 157-207. Springer, 2006.
Litu Rout, Alexander Korotin, and Evgeny Burnaev. Generative modeling with optimal transport maps. In International Conference on Learning Representations, 2022. URL /ttpps::/operrreeiee net. http:/frummmiccc (tt.oouummiid-5300222333555
Maziar Sanjabi, Jimmy Ba, Meisam Razaviyayn, and Jason Lee. On the convergence and robustness of training gans with regularized optimal transport. Advances in Neural Information Processing Systems, 31, 2018.
Filippo Santambrogio. Optimal transport for applied mathematicians, Springer, 2015.
Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge university press, 2014.
Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. Diffusion schrödinger bridge matching. Advances in Neural Information Processing Systems, 36:62183 (6662183-62223 662223,, 2023.
Shashank Singh, Ananya Uppal, Boyue Li, Chun-Liang Li, Manzil Zaheer, and Barnaáás Póczos. Nonparametric density estimation under adversarial losses. Advances Neural Information Processing Systems, 31, 2018.
14

---

Published as a conference paper at ICLR 2026
Luwei Sun, Dongrui Shen, and Han Feng. Theoretical insights into eyclegan: Analyzing approx- imation and estimation errors in unpaired data generation. arXiv preprint arXiv:2407.11678 2024.
Amirhossein Taghvaei and Amin Jalali. 2-wasserstein approximation via restricted convex potentials with application to improved training for gans. arXiv preprint arXiv:1902.07197, 2019.
Alexander Tong, Jessie Huang, Guy Wolf, David Van Dijk, and Smita Krishnaswamy. Trajectorynet: dynamic optimal transport network for modeling cellular dynamics. In International conference on machine learning, pp. 9526-9536. PMLR, 2020.
Ananya Uppal, Shashank Singh, and Barnabás Póczos. Nonparametric density estimation & conver- gence rates for gans under besov ipm losses. Advances in neural information processing systems, 32, 2019.
Adrien Vacher and François-Xave Xavier Vialard. Stability and upper bounds for statistical estimation of unbalanced transport potentials. arXiv preprint EeXiv:2203.091433 2022.
Adrien Vacher and Trançois-Xavier Vialard. Semi-dual unbalanced quadratic optimal transport: fast statistical rates and convergent algorithm. International Conference on Machine Learning, pp. 14734-34758. PMLR, 2023.
Francisco Vargas, Pierre Thodoroff, Austen Lamacraft, and Neil Lawrence. Solving schrödinger bridges via maximum likelihood. Entropy, 23(9): 134, 2022.
Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2008.
Gefei Wang, Yuling Jiao, Qian Xu, Yang Wang, and Can Yang. Deep generative learning via schrödinger bridge. In International conference on machine learning, pp. 10794-10804. PMLR, 2021.
Yujia Xie, Minshuo Chen, Haoming Jiang, Tuo Zhao, and Hongyuan Zha. On scalable and efficient computation of large scale optimal transport. In International Conference on Machine Learning. pp. PMLR, 2019.
Pengchuan Zhang, Qiang Liu, Dengyong Zhou, Tao Xu, and Xiaodong He. On the discrimination- generalization tradcoff in gans. In International Conference on Learning Representations, 2018.
15

---

Published as a conference paper at ICLR 2026
A PROOFS.
In this section, we provide the proof' of our theoretical results:
Theorem 4.1 A..1
which decomposes the recovered map T error into several approxima- tion and estimatinn subterms depending the values of the optimized functional L.
Theorem
A.2
which upper-bounds the estimation error with Rademacher complexities
function c classss F, H.
which demonstrate that the approximation enror could be done arbitrary small under properly chosen classes of Neural Networks.
Theorem 4.3
{A.3) and Theorem 4.6
A.4
Theorem 4.9 (SA.5) and Corollary 40.0 (5.55) which establish upper bounds for the gener- alization error ( and show that it can be made arbitrarily small by choosing the appropriate classes of functions and sufficient number of samples.
Before starting the main proofs, state the following auxiliary lemma Lemma A.1. Let A, B g
R be some functionals functional space g. Then inf A(g) inf gEg
[Bg) S sup [A(g) B(g)):
gEG
9CG
sup A(g)
sup B(a) S geG
sup gEg
AA(g) - B(g)).
gEG
Proof. In what follows, we prove the first inequality in the statement of the lemma. The second inequality could be derived in a similar way.
Without loss of generality, assume that
inf A(g) > inf B(g) g
inf A(g) 9Eg
inf B(g)
inf A(g) inf Bgg).
g
9Eg
sE9
DE9
Let pick and consider ge 9 such that B(g.) inf, B(g) 6.
Note that
inf, A(g) s A(g.). Therefore:
inf A(g) inf B(g) < A(g)) - B(g.) te
9E9
gEg
< AA(g)) B(g:)) +
- sup A(g) - B(g) +e.
9Eg
Taking the limit 0 in the last inequality finishes the proof.
A.1 PROOF OF THEOREM
First, we upper-bound the error in estimating transport map via duality gaps analysis. Our theorem below borrows the main ideas from (Makkuva et al. 2020, Theorem 3.6), but has own specificity, since it deals with transport maps that are not necessarily given by gradients of convex functions. Theorem A.2. Under the condition of theorem
4.1 it holds:
T - 7
4
(6 TTh &z((²))
1.(p))
B
L(@" T is the inner error, and E2(RR) def max L( T) min max L(p, T) is the outer error.
where E1 [.
def
C( T)
T
T
Proof. We introduce Tza del
arg max, L(G ( T). Then,
EGGR, R( / ((x))) -6" (x)) p(x)dr I (c..T"())) - -f"(("(<)) p(z)dz.
16

---

Published as a conference paper at ICLR 2026
Consider B-strongly concave fz(y) def (x, 1) (G) Then,
Ei( T = / (1((Ta(())) Ia (x)) (x))))())
(21)
Note that (z)(2) is the maximizer of Sx (cf. Remark (for any 7):
Thanks to ß-strong concavity of fa it holds
B [[(()) - T (x)
fz - 2
(22)
2
Combining
21 and 22
we get:
E T
T
TgR (P(p)
(23)
Now we move to (, We have the following:
E2(² C(B", T(p) ,T""
/x (r, Ta (z))p(x)dd
6" " (2a(()))) + / ""(y)aluidy
y
- -fx (x, (z) (T(T(z)) ('((((v)dd
since u=
/ -0 / ""(v)al(dy J (o, (o))))))
(24)
-f
/ (T)("(())( - (z. T (x))p(())
(25)
S (2, (( p(x)do / ((,, - G (I (x))) p(x)dz,
=s((Ta((z)
=fa(T((w)
/ [fc(TTa(=)) (T (z))lp(r)dr
(26)
2 2 / [TT(ss) (1)( p())dr.
3
(27)
where in transition from fxx (T* (x))p(x)) in transition from (66 to (27 we apply property (22) for T given by true map T
24h
25 use Top 9 which yields Jy ("())luv)dy
Application of the triangle inequality for 23 and finishes the proof:
A incq.
| - 1 La(p)
T
T Taa Famlle2(0) + || - ("((((()))
2
2 ( C TiR) + v
2
3
(0(2R.AR) + E2(6R))
Now we proceed to the proof of the main theorem. To get the final result, remains to decompose both E and & errors from the Theorem A.2 into estimation and approximation parts.
17

---

Published as a conference paper at ICLR 2026
Proof of Theorem 4.7
From Theorem A.2
we have:
E
TR T
12(6) S 2 E E
xx.
(
(R)
E. E3(R))
(28)
x.y
X.
For the inner error E we have:
E, ,
T = max C(p", .T) L(G", $(@".T"! )
T
max ((@R, .T) max C(FR T)
+ TET
max C .) - C(
TET
S max PEF
max [ T) - max L(p,, T)
+
max C(@R. .)) L( ,).)
TET
TET
Taking the expectation of both sides in the inequality above yields:
E E s +F
(29)
For the outer error &g we have:
E2( = max L(G", T) min max L(p,77 T)
T
T
max L R T)
L( R T) + max ( T) TET
max
min max C(p, PET TET
TET
+ min max ((p, T) $EF TET
min max L ( T) PeF
min max C(p, T) PEF
min max L(p, T)
T
T
ip
T
(30)
max FEF
max ((p, T) m L(
max C( max max C(p, T)
TET
TET
PETTET
max
max [(p,T)
max C(p, T) T
min ((p) -((x")
(31)
PET
TET
PET
EA
(F,T)
+
max L(@R
T) min max ((p, T) PET TET
+ EA (F, EA (F),
(32)
TET
where in transition from Taking the expectation of the inequality 32
30 to 31 we apply Lemma A. for dashed terms.
results in the following:
E E2 E (F,T) + EE (F,T, (F, (du
(33)
The combination of 28
29
and 333 finishes the proof.
A.2 PROOF OF THEOREM 4.
Our theorem uses some standard notions from learning theory, see, e.g. "Shalev-Shward & Ben-David 2014 $26). We recall them for the convenience. For class of functions & y R and probability distribution y, the representativeness of sample Y == [VI..... ..M ~ of size M:
Rep (Y) sup PEF
/ ((v)a(M)dy
1
Σ p(ym) m=
M
(34)
M
In turn, the Rademacher complexity of the class F w.r.t. the probability distribution g and sample size M is given by:
p(ym ) PEF
R.
sup
Σ
(35)
M
where {ym m=1
1 9 are mutually independent, o {am M are mutually independent Rademacher random variables, i.e., Prob(an. 1) Prob -1) 0.5. and the expectation is
18

---

Published as a conference paper at ICLR 2026
taken with respect both Y, J. The quantities (35) and 33) have well-known relation (Shalev
Shwartz & Ben-David 2014 Lemma 26.2):
ERep (X) S MM
(36)
where the expectation is taken w.r.t random i.i.d. sample Y q of size M.
Now we proceed to an auxiliary statement needed for the proof of our Theorem 4.2 lemma, we upper-bound the difference between the original and sample-based functionals L and with representativeness numbers w.r.t. classes F, H.
the next
Lemma A.3. Let X and Y be training samples from and q, respectively. Under the conditions of Theorem it holds:
sup sup [((, - [(P,T)) & R (X) + Rep (Y), PEFTET
(37)
where Repx (X) and Re (Y) are representativeness 34 of the corresponding samples w.r.t. corresponding classes.
Proof. Let us write out in detail the right-hand-side of 37 and regroup the terms:
sup sup [C T) [({,TT)
SEF TET
/ T(x)) (((())) / p(v)a(o)dy
sup PEF TET
N
M
[an, - (((((x))) - M
Σ
3 (vm)
m=1
/ [e,T(x)) - (((())) p(x)de
1
N
sup PEF TET
KT T(x))) (((((xn))
N
1=1
=h(xx)
+ /ʃ plu)a(u)dy
M
Σ
(Vmm)
M
m=11
< sup sup NEH
f Mooo)(o)de
1
Σ h((n))
+ sup PEF
/ p(u)a())dy
Σ plym)
N
M
Repx.,(X) - Reps,g())
Now we proceed to the proof of Theorem 4.2
Proof of theorem
4.2 We start by decomposing E and & into subterms:
EP (F,, N,M)
E X.Y
L( ( max C(@ .))
TET
E |(( R. L(@RR TTR)
(38)
+
[(G TR)
max L( .T) TET
(39)
EE (F,
E
max L(@R T) TET
min max C(y,, T)
X.Y
Pef TET
max L(@R RR T) 2(0B TR/ TET
E
(40)
+
[(@R 7R)
min max C(pp, 77| FeF TET
(41)
19

---

Published as a conference paper at ICLR 2026
Now analyze the subterms from the equations above separately. For simplicity, omit the expectation " sign.
X.Y
Term 411
LI@R TR) min max L(p,, T)
min max IEF TET PEF TET
C(p, T)
min max Liv,, T)
(42)
PEF TET
Notice that m C(p,, T) and maXTET ((p T) are some functionals, depending on yp (denote them as 1(pp) and (55)). Then we can apply Lemma A.1
min (pp) min PEJ
min (19) PEF
max PEJ
((p) - [((p)
max PE
max [(p, max
(43)
TET
TET
Applying Lemma
again to the internal expression, we get:
max PEF
max ((p,
max ((),T) TET
max max S
((p,)))-((p)))
(44)
TET
PEF TET
Terms
39
40
Similarly, we apply Lemma
C(p" Th) max TET
L(Ø", .T) .T)
max L( T)
L .T)
TET
TET
max
C(ØR, T) L(G T)
(45)
TET
Finally, get that all four terms
38
are bounded by
max,er maXTET [((.TT) T) C(p,, T) Therefore, by applying Lemma A.3 we get:
& (F) + EL. (F, N,N, & E N,Y
max max o TET
C(p,, 1) C(p,, 2
4IE ReRH. (X) + 4IE Repy (8)
8R (H) + (R..
which finishes the proof.
A.3 PROOF OF THEOREM 4.3
To begin with, we recall some standard definitions and notions used in this subsection. Given function C()), its Lipschitz (semi)norm is defined as:
(p(()) s0y')
sup
w Ily
y'112
g.y/ ey
(Continuous) functions with bounded Lipschitz norm are called Lipschitz continuous and denoted Lip()') c C()).
Now we proceed to the main content of the subsection. The next auxiliary lemma states that the differences in the functional values could be controlled by the properties of its arguments.
Lemma A.4. Let vp E Lip(Y") and Tí, X + y be measurable mappings. Then,
[i( - C(p, 1b) & (sup ||ا|| + [|<))e)))) || Tellz.
20

---

Published as a conference paper at ICLR 2026
Proof.
Consider the seft-hand-side of the desired inequality:
(i) - ((p, = L ((<x) - - -(xxTT(x))
- ((Tb(x))) fo(a)do)
Jensen iney.
J ((x) ((T) - ((x,T2(x)) - ((Ta(z))) plz)de.
(46)
Consider the expression under integral sign:
((o, (x)) - ((()(=)) (((x,T2(<)) - (T2(())))
< (c. + (x)( -
< ||||
- ||elاzi - (47)
The substitution of 47 into 46 finishes the proof.
Corollary A.5. Let T arg maXTET C(v,, T), TG arg maxy ((p, T). Then,
max L(p,T) max TET
((p, T) S ( sup |
((pl)))))) T Telle,te)
xex
The next auxiliary lemma upper-bounds the (p) nomn between maps which are optimal for given dual potentials.
Lemma A.6. Let $2 Lip(J/) be B-strongly convex. Then:
1 [T - Tealle, S 3 |llp pellzip
where Tú, = arg max T): = arg max L(p2:
Proof. We pick x E X' and consider function y + (z. 3) 41 (g). Then:
* TTp (Ir) is the maximizer of y + (x, - 4 (v) (see Remark
y (x. şi (w) is a B-strongly concave function.
By the property of strongly- function, we have:
3 1T (z) (x) (z, (T, (x)) 61 (x))
Tx, (x)) 91 (Te) (x))
8 11T (x) Top (x) & (T)) - (x)) (z, To, (x)) - (x, Top (x)))
(48)
Similar inequality holds for 422
[x)())) - 62 (Tpz (x)) - [T (z) - (z.Ta, - 42 (Tp, _
2
/ 1TT (x) - (x)) 22 -20
- (r, Toi (z)).
(49)
Combining inequalities 48
and
49
we get:
3 1T (x)
(x) Tx2(2>)
< [4p s22] (Tp, (z)) [4p -22) (T (x))
S || rellzup [T (x) - Typ (x)
i.e. [[Tp, (x) - (x) < 1 - -22l Taking the expectation w.r.t. probability distribution
p(x) completes the proof.
L
The next lemma is a technical result, which allows to restrict the output of mappings NNN X + R parameterized with neural networks compact support space y' with help of projection function.
21

---

Published as a conference paper at ICLR 2026
Lemma A.7. Consider the projection operator projy :R y. defined
proiy(r) = arg min |1 2
yEy
Then, Vri, E R it holds:
(ro) S ||n r2l12-
(50)
In particular, projy is a L-Lipschitzffuccinn function.
Proof. To begin with, we prove the following technical fact.
Part I. Technical fact. Let y E VVr E P yv. Let y+ = (r) Then:
(y Ma 1/ r) 2 0.
(51)
Consider variable [0, 1], and ys (1 3)y. + By. Note that yo E V, since y is convex. We have the following:
llys r12 y
y+ + y/ - r112
B - y. 12 + 23 - y. y. - r) + 1ly. - -r122
By definition of projy it holds: |/y, r||2 2 ||y -112- Then,
B² fly
1-112 2/3
Ve. V+ r) 2 0:
(y y., V.
r) 2 - -32 y. l1 (divide by 233 O);
B
(y y. V/
2 lly y, 1
The last inequality holds for every 3 0. Therefore, (51) holds true.
Now, we move on to proving the statement of the Lemma. We consider different options:
(i 71,72 e):
(ii) T1 E ) and r2 ( (equivalent to r2 E ) and rı E (R))))))
(iii) 71, E D
Note that case
i trivial. We proceed to cases (ii) and fiti
Part II. Proof of the Lemma; case (ii)
For ease of perception, introduce 1/1 := projy (r1) = 11 E V. Let y+,2 projy (r2). Then:
||r rel12
1/ r2l12 = |1 - V..2 + y,.2 - r2112
V+.2 r2l13 + |yy - 2 + 2 (y1 U4, T2) T2)
20
-llpooj (roiy
20, sec 51
(m))
Part III. Proof of the Lemma; case (iii)
Let v.,1 = projy(ri): n (r2). We have the following:
|||] - 1 = ||| - y/.11 + Un,1 V/ + V/.2 - rell
||n Wr.1 + V+.2 rell + Il Br.2 13
prop proj (r2)11
+ 2 (r1 - V.,1 Un.1 - y4 +2 (11,2 12 r2. v4.1 - 34
20, sec 51
20, 5T
> |projy (r:) - (reo)
22

---

Published as a conference paper at ICLR 2026
Now proceed to the proof of the main theorem statement.
Proof of theorem 4.3 First notice, that from compactness of F it follows that F is uniformly bounded, i.e. AL E lk Vip F l1 L.
For fixed arbitrary small E pick ò1 2(sup.cx [1]]++ follows that there exists finite ó1-covening of F:
80
From the definition of compactness,
Vo, > 030 I(61); 3p...... EF: F F
(52)
where J; (:: & 114 9i Pillzip S 661
Pick 02 2(suprex [
According to (Kidger & Lyons
2020
Theorem 4.16) for each Tea.
there exists neural network T (622)
such that
T
NN(22)
Tea
S 82 for arbitrary small
(1(r) I
52. Thanks to Lemma A.7
projy Top
NN((6))
To Li(p)
Tpp (((p) I 82. Let's
choose class of neural networks Tus(o) such that it includes all the functions Twt (62)) Then for such class:
(N66)))
y..., pm
max C(pk,, max
C(pk, orojyo)) S max ((pk, C(pkk projy projy T67x)(62)
(53)
Now we are ready to bound inner approximation error.
Consider yp E F. Then yp belongs to one of the sets of finite coverage Fk. According to lemma A.6
have I|T - -Tpll.. = Pllzop S &
We also can approximate Tę. by the neural network, so we can do with Tig:
+ 1T -Tell S 82 + 81
projy (2(6)) Top
proj o oT
-Tpx Li(p)
3 (54)
LT(p)
Then it remains apply Lemma
A.4 and use the uniform boundness of F:
max C(p,, T) o (6N)() Pa
sup wee
||=||| + 110/llepp)
|Projj o (.NN)(62)) T (T(R)
||>||| + 2) (
62 (02+ + 01 = =.
sup
(55)
Due the arbitrariness of the choice of ip, we obtain
max P
max [(
max TETw((1))
C(p, oroj SE
(56)
A.3.1 PROOF PROPOSITION
Let us denote &s as a K-layer ICNN Amos et al. 2017) with skip connections of the form
21 (1() s(a1y bz)
2 1 (y) = ((W412((1) (v) + a:+1y + b.+1)
(57)
2x(2) = s(wx2k-1(1) axy bx),
23

---

Published as a conference paper at ICLR 2026
where w; Wi, E Ar, b, E B,, and for i-th layer W, (W1 0), A, and B, compact subsets of IR D D; (nonnegative weights), D and I2 P respectively for any i-th layer, convex and non-decreasing activation function; D (ICNNs have scalar output). If s is ReLU, then f is a neural network from universal approximation theorem for Lipschitz convex functions (Chen et al. 2018 Theorem 1). Further for convenience we denote h; - (y) + a.y + by;
p(yss) = (w;, dj; bz)l = 1,..., K}: ; P = {(w, at, bi)lu; E W;, a; A,, E B is whole parameters space; Pn = {w1, (1, ...... WK: (K, bx} J are parameters' "names".
Lemma A.8. Let be smooth function. Let E Pn be parameter of ICNN Ps. Then 4g 0 3e such that |pp - -p'pp2 & 1lv. - -
Proof. Notice that ys differentiable. Then llp. max [V)l2
Consider E(y.p) def Vello=p()), i.e, & the gradient (w.r.t to input y) of ICNN 9, with specific value of parameter In particular, s"v,p") (v). Note that yp is smooth (as the composition of smooth functions), both w.r.t input y and parameters Therefore, E(y. p) is continuous. To finish the proof we are left to note that E(v. p) is uniformly continuous (y. p are from compact sets).
Lemma A.9. (ICNNs (7) are totally bounded). Let F be a class of K -lyee ICNNs of the form 57 with smooth activation Then totally bounded with respect to the Lipschitz norm.
Proof. Pick arbitrary 0. From Lemma A.8 for each parameter p E Pas we can choose Ep such that:
5 [Pa]
1p- p2112 < Ep llp= P:llzip
where [Px] is the number of parameters in the neural network (it is finite). Detine == minpeP, Ep- Each parameter p belongs to a compact set P; we consider a (finite) e-net & that covers P. Then, Cartesian product
3
X
Ep
PEP
defines the (finite) e-net that covers the whole parameters set P. Within each "ball" E E, different combinations of parameters D.i.pp.....PPp Pfppp (ri Pp' ....... Ppp)) s are s-close to each other point-wise, i.e., |p p ||2 c. By subsequently substituting pt with pp i 1,2,.... [Pal we obtain:
l1p; 1l < <6
iee., E defines 5-net for s.. Thus, F totally bounded with respect to the Lipschitz norm.
A.4 PROOF OF THEOREM 4.6
Below, we will use the notion of B-smoothness. Function & : y - is called B-smooth it is continuously differentiable y. and its gradient Lipschitz continuous with Lipschitz constant B:
( v<((u2)12 B Il vell2 Vy1, E
We proceed with several auxiliary results needed for the main proof.
Lemma A.10. Let is B-smooth on y. Then there exists > O such that yp is L-Lipschitz, i.e. I I L.
Proof. By fundamental theorem of calculus along segment y to y' we have:
( x (v/) = L (Vi (v' + & (y v')), v/) dt.
Applying inequality get:
24

---

Published as a conference paper at ICLR 2026
(K(() -00)) L 11v (v' + + - 10--y))) 112 Bl - 1/ dl.
(58)
Given that Vip is continuous (because yp is B-smooth) and y is bounded, then there exists L defined as L = yey yey [V)())l22 < oo. Replacing in (58) we get
lp(v) ( - L lly
v'122
which concludes the proof.
Lemma A.11 (Property of convex conjugate sup norm). Let yI,, sp2 E C()). Then,
ক - 22 S ll sP2 loo.
If 91 and i22 are convex, then the inequality becomes an equality, i.e., P0 12 P2llox = ||,p1 - 102 loo.
Proof. We pick E X. Then,
(
(2())
max NEY
(r. v) -11(1)] max (22, (2 NEy
LemmalA.I
max yEy
KerT) (1(())
(L-0) (22211)
Taking the supremum w.r.t æ in the inequality above yields iii and 222 convex, then:
1211 S |451 - p2llo0.
|| 402 bl
|
422
P
222
which proves the remaining statement.
Lemma A.12. Let & be a set of convex functions on compact domain y C RD and uniformly bounded with respect to the Lipschitz norm. Then for any = >> 0 there exists such constant L = L(E), that any & can be arbitrarily closely approximated by an L-piiihhit function, i.e., for any e one can choose Lipschitz continuous function yPI. E Láp()),, Il & T. such that
11g -ppllo S.
Proof. Consider yp &b and and take B B(s) (recall that yp is convex conjugate of :p:
supy Ilvll 2F
Consider the following functions
10113 2B
11
(xx) def (xx)
def
PL
2B
Note that &b is g -troongyy convex. Therefore, by Kakade et al. 2009 Theorem 6) Pr. is B-smooth. Applying Lemma A. we get that yr also Lipschitz with constant -= I(e) (note that L does not depend on particular thanks to uniform boundedness of d w.r.t. lischitz norm).
Finally,
11-11
| 2B
11<p
sp
Lemma
4
8
sup
6,
2L,
which completes the proof.
The next Lemma improves on (Chen et al. 2018 Lemma 1),
Lemma A.13. Let & be a set of convex functions on compact domain y C. R and uniformly bounded w.r.t the Lipschitz and sup norms. Then for each > 0 one choose K such that Vip + can be approximated by maximum of at most of affine functions with parameters on compact sets. That is Vip E & there exists ply) max; I.... {ui y + bi), such that 1|<p @l < and < K. t4g M, by B, where M C T and B C TR. are compact (and do not depend on particular sp).
25

---

Published as a conference paper at ICLR 2026
Proof. Note that y is compact. Then there exists finite 4r. net & of yf [E| = K. In what follows approximate convex function yp with linear function on each of the E iL balls.
Note that 4 is uniformly bounded, and there exists the global Lipschitz constant L: || S L.
For each E. ball & E & let us choose an interior point y/ where yp is differentiable and consider langent plane μ3 y + b, at this point. Note that cp(x') = nu y + b, and Vy E y : (p()) 2 14 y + b, These planes are also L-Lipschitz, therefore for any other point v/ E si it holds:
|x(x)) (p())1 - 1| wlll -
e
2'
[1, y3 + b.) (n, vs b.)1 L 1|
v - 2'
Summing up these inequalities, we get:
(e(u²) (n. oス + b.) [+<(v)) (N v! + b,) S c,
i.e., on ball & our function could be approximated from below by the corresponding plane n4 + by-
Now we define ((v) def max {p, ( y + b,} y + R. where (Mi, b:) correspond to each Obviously,
(e(v)) [()) (<v) (vv) Vy E..
In remains to notice that by property of Lipschitz functions we have | S L. turn, the compactness of parameters b; follows from the uniform boundedness of + w.r.t. sup norm (ll - llso).
Taking the advantage of our Lemma A.13 we can formulate an improvement over (Chen et al. 2018 Theorem 1):
Corollary A.14. For any E > L, 0, > O we can choose K and compact parameters set P such that for any convex function yp with || S C, lk<pllap I L defined over compact domain y there exists an ICNN (7) Po with depth and width limited by K, with weights p E P and ReLU activation functions that approximates it within
11p Polloo S.
Proof. Follows the proof of (Chen et al. 2018 Thm. 1), but utilizes Lemma (A.13) instead of (Chen al. 2018
Lemma 1).
any bounded set. Then Ve > 0 and parameters space P we can choose n n(e, P) E N such that two ICNNs (57) ss., and (pa with coinciding weights p((p...) ) = p(vp,) and different activation functions sn and respectively
Lemma A.15. Let 8n.
close by co-norm:
lle.. PPlllo <E
In particular, 8n could be chosen be CFLU, activation functions CFLUn(x) max(0, min (0,
- (1)= o is ReLU activation.
Proof. First, let's notice that for fixed compact P from 57
compact input domain and bounded activation functions, output of each layer in ICNN belongs to compact set, i.e.
max p(PiEP.WE)
(1x(()))o < Const
Let's denote layers two neural networks s.,. and &o as 2 and zg. We prove the statement by induction. Consider the difference of the first layer outputs':
|| (v) = (v) ||s, (4i"
ah2 (l))
26

---

Published as a conference paper at ICLR 2026
As weights are equal, then h (y) (y). Recalling that for p P function (of parameters and inputs) hi" (y) takes values over compact set, due to uniform convergence of Sn, we have
P(P)EP.BE> 1=f <1)]l>>>
Consider then difference of k-th layers
|| (v) - 2 (v))l = ||sn(1 (v)) 0 (b) =
( (y)) (n)(((()) ao + (lan(h (b
(59)
Consider the first term of (59 Note that || (v) - hg (y)ll = um (2 Therefore, due to continuity of sn and boundedness of wy E Wx it holds:
[|s (x)) sn (v (1) + 0.
pEP,zEN'
The second term (12 - (1)))l uniformly tends to zero because of assumption of our lemma 5n =
Finally we get that max P. 2 (x) tion, we have uniform convergence for the ICNNs itself:
2% =x)(2)) + 0 for k-th layer. And, therefore by induc-
max PEP,VEY
119 (B) (b)ll + 0.
Lemma A.16. Let & be class of convex functions, uniformly bounded w.r.t Lipschitz and sup norms. Then for a fixed = there exists a class F of ICNNs 57) with CELU; activation totally bounded with respect to the Lipschitz norm, such that
Vi E 4 3p. F 11lp-paa Polloo <.
Proof. By Corollary A.14 we can choose ICNN for of the form 57 with width and depth bounded by
K K(e) and weights from compact set of parameters P = ((e) approximating within 2.
E 2
l14o 01
By Lemma A.15 we can choose n(e, P) such that replacing ReLU activation by CELU,, will
lead to controllable changes yo:
11 San
Summing the inequalities above we obtain:
llya.. pl1o0 <.
Note that CELU, = CELU)(n)) Therefore, CELU, activation functions in ye,, could be substituted with CELU, by proper modification of parameters set P (which does not change the compactness of P.).
Now we are ready to prove the main Theorem.
Proof of theorem 4.6 By the assumption of the theorem yp" Consider
arg min, C((p) is B-strongly convex.
11.13
-/ 8 def
6
2
Note that - is convex. From Lemma A.12
can find so Lip()) such that | s
L L(e) and:
- splloo <
7'
27

---

Published as a conference paper at ICLR 2026
According to the exists class Fienn F Ficnn(e) such that find there neural network LL" close to any L-Lipschitz function:
ll 12
(60)
Obviously, 922" also Lipschitz since is a composition of piecewise linear functions. Consider
Pr. P1 + 1-1111
2
Note that 2L is B-strongly convex. Then,
11p- 2 - |1:0 - 10o S - + +1101. - 12 1m0 S
Let us check that 2 delivers the desired bound.
(((9)) -(0") / (1(x)(( + I, P..Waalidd (S, (0) I 40 (((a(o)))))
(x)xx) (x)))p(()) + / (0'(0) (
x
/
80 oL |g p(z)do -
ri ,(y)dy
=||
- by A.11
Therefore class F= {pp + 3 111:18
2 pe Fromn) satisfies the condition the theorem.
Proof of corollary 4.7 It remains provide some class F, which contains Pi. We can choose sufficiently expressive class Fiena and then define F such that E F has the form x +> ypa (2)
ا where yPg E Fienn. For compactness of F by Lipschitz norm we can bound all weight matrices by second norm.
A.5 PROOF OF THEOREM 4.9 AND COROLLARY 4.10
Proof of theorem 4.9 9
Let's bound all the terms from decomposition in theorem 4.1
step by step.
From corollary 4.7 we get such class F F( 3s () that & (F) < As The second step is apply
theorem 4.3 which gives us T = T( 1 F). such that 3 (F,T)
<
de Finally, applying theorem
bound remaining terms
E (F,,,NN + E (F,T,N,M) & 8Rp (H) + 8R).
Proof of corollary 4.10 Our goal is obtain bounds depending on sample sizes for Rademacher complexities of function classes H = H(F, T) and F which appear in the inequality (19 of Theorem (projy T.T Tus): Tus is class of feedforward neural network with
4.9
Recall that:
bounded norms of weights, limited width and height and Lipschitz activation functions ReLU;
every function & E F has the form x + (2) + ||=|l where ye belongs to the class of input convex neural networks Frenn- Here we additionally assume that neural nets from Feem have bounded norms and CELU activation functions. Let T be a more general class of feedforward neural networks with any Lipschitz activation functions (2x). e.g., ReLU, CELU and others, s.t. (0) = 0. Note that the Rademacher complexity of class T' is known to attain an upper bound: Rp.N & 50( / where O(-) hides the constants which do not depend the sample size N, see, e.g., Goooooich al. 2018).
28

---

Published as a conference paper at ICLR 2026
Step 1. We start with obtaining an upper bound for Rg (F). From the well-known monotonicity property of Rademacher complexity, we get that
Fienn C CT = Rg (Frena) S Rg RR) S 0 (
VM
The complexity of the constrained quadratic functions x + 1110118 is also O O(7M)) It follows from their representation using the Reproducing Kernel Hilbert spaces (RKHS), see, e.g.. (Mohri et al. 2018 Thm. 6.12). (Latorre et al., [021] Lemma 5, Eq. 24). Then, from the additivity of Rademacher complexities it follows that the complexity of functional class F is also bounded by O VM
Step 2. Below we prove that (H) attains an upper bound O V Recall that class H consists
of functions h which map x + (roo)(((())) - ( (T(x)) where T Tab T
F T'. Note that the function وp o projy T is composition of feedforward neural networks with the Lipschitz activations from (pluus additional intermediate I-Lipschitz "activation" projy. see Lemma A.7). and, thus, belongs to the same class T' By monotonocity property, the complexity of this class of functions is bounded by O which follows from the first step of this proof.
It remains to show that the complexity of the class of functions + (Ir, projy (Txx)) for T E Tub is also bounded by O 1 To prove this fact, we consider empirical Rademacher complexity for this class of functions (T") defined as
R def 1
E sup SET"
1 E sup TET
Σ
nn ( (2nn)))) }
(61)
N
Here the difference with the classic Rademacher complexity definition (35) consists in the fact that the expectation is taken only w.r.t. the Rademacher random variables fo J. Importantly, can derive (35) from (61) by taking the additional expectation w.r.t. random samples X = (I1,,.,, (
For simplicity of further derivations, we introduce function T" RD + R2D s.t. Txx (x, Tw)). def (x, (r)) Then (61 can be
Besides, we consider function / X x RD + R s.t. h(z,r) rewritten as:
N
R Rp.. (T")
E sup Σ (h (Tr N TET" n=1
(62)
N
We start with proving thet h is G-Lipschitz function. Indeed, let æ1, 22 and ri, belong to X, RD, respectively. Let y1 = (ri))(r)). 1/2 = (rojy(12)) Recall that X, are compact subsets of RD which means that for all x E X, y E y they have bounded norms: max(] l1vll2) S R. Then:
[1(11,r)) - h(22, 12) = (II, B1) - (x2,, 32 = [ 31 322 + (z) x2, 82) <
(4118/ - 12 + [x - x2, x2, 12 s ||
1/ vell2 + |1 112 || - x2 11 <
dt 50
R² (11v) vall3 ||ا| x2 12)
R² (1lr r211 + || - x2 l12)
R²
[1 v1) (ar2. y2
This allows to apply LLi et al. 2023
Lemma B.1) (taken from Maurrr 2016)) to get
Maurer
2016
1
N 2d
T' (z } 5
62
V2G
E auttaaaaa
sup
N
TET"
n=1
N
N sup
22G
E Nd
Inle Tne
+
1
V2G
E
Te One J. (63)
N
Na
N
Na
TET
n=1
m=1
Applying (Lei et al.
2023
Proposition 5.5), we get that
N
Ti(en)one } S svac, C2. C2. (cf
N +
E
sup TET
Σ
|, (64)
t=1
15i<JEN
/=1
29

---

Published as a conference paper at ICLR 2026
where C1. C2 are two constants which depend only the number of layers in neural nets from Tub and bounds on the weights. Besides,
[nn]
E
nit
(=1= t=1
Using these observations, we get
N vacivvac C2. E (1f +
1 N
v2
63
| 11
(65)
ISiRjSN
In remains to take expectation of 65 w.r.t. X [f1,...,N) p. Before moving forward, note that [xy ~
are i.i.d. vectors.
Additionally note that since working the compact subsets of R for each i + j:
B(o xz)² < El
2 E R
At the same time, for each we have | S
R Then we get,
Jensen's inoq.
V2dGC
E x #11
C2. Σ
(xTT))2)
+ Σ
|
ISKjSN
1=1
1
/2dGC
Co C2 Σ (xTaz)2)
| 112
N
ISiSINN
(C2- Σ
2dGC
E(x 22
(66)
N
ISKUSN
SRa
SR
1
V2dGC
(C2.V
N(N 1)
R2 + NR
= =0
N
2
It completes the proof.
B EXTENDED DISCUSSION ON THEORETICAL ASSUMPTIONS & LOWER BOUND ANALYSIS & PRACTICAL RELEVANCE
Discussion on the 3-strongly convexity assumption. It seems we can not get rid of this assumption. Now we explain the roots of this below:
If we allow in Theorem 4.6 that yp" is not 3 Lipschitz, still may achieve L((pt) - L((p*) but for B = Ce, C' is some constant.
This makes Theorem 4.1 not useful, because we restrict L gap by & J. i.e. can not achieve arbitrary small gap.
Theoretically, B-strong convexity for yo holds where the OT mapping between distributions is regular enough. By (Kakade et al. 2009) and our Lemma 10 means that T is Lipschitz, i.e., does not push close points far away from each other. We assume this holds true for reasonable (smooth, supported on the same set) underlying distributions.
Lower bound analysis. To start with, we need to explain how we position our paper compared to other works which conducts statistical analysis of OT (primarily, (Hütter & Rigollet 202T). (Divol
Gunsilius )) Overall, the general framework of these papers is as follows:
et al. 2025).
1. They place certain restrictions on source/target distributions and class of functions in which we looking OT mapping;
30

---

Published as a conference paper at ICLR 2026
2. Under these assumptions, they derive (minimax-optimal) rates;
Then they discuss or construct some family of approximators which some extend satisfies the
assumptions.
This framework provides strong theoretical foundation, though one practical consideration is that the proposed approximators (which achieve the optimal rate) may differ from those typically adopted in real-world applications.
In our paper, we take a different approach. We do not place restrictive assumptions on the optimized class of functions (we pick MIPs, ICNNs), and analyze the minimax OT solvers, which seem to be quite popular in practice, see $2 for the list of the methods which are relevant to us.
We think that in our setting much harder conduct study convergence with more sharp rates. One reason is the complexity of saddle-point nature minimax solvers (the aforementioned papers study just minimization), the other is our weak restrictions on function classes. At the same time, we emphasize that our work seems to be the first which conducts statistical study of popular and algorithm used recover OT maps.
practically-appesn
Practical relevance. In our paper, we study theoretical properties of certain Optimal Transport solvers. We insist that the ability to model OT maps rather than arbitrary push-forwand maps an important and practically inspired field of research. One of the most intriguing applications of OT is unpaired domain translation of images, see, e.g., (Korotin et al. 2023b; Shi et al. (023). Here, thanks to the 'optimality" of the learned image-to-image mapping. the content of the transformed images remains similar to that of the originals. This could be crucial for image enhancement and super- resolution applications, for example. The other solid application of the OT is computational biology (Tong et al. 2020 Demetci et al. 2022 Bunne et al.) 2023) (manipulating single-cell data). Here, the OT appears by nature of the problem; turns proliferation of cell populations.
be deeply connected with the development and
Importantly, all of these practical cases have inspired the development of a variety of minimax OT solvers (close to our theoretical analysis), see the list in the end of $2
C ADDITIONAL EXPERIMENTAL ILLUSTRATIONS
C.I ESTIMATION ERRORS OF BASELINE METHODS
In this section, we consider three non-neural
0.5
DIM=2
DIM
000-0000-000
000-0000-900
baseline methods: barycenter translation esti-
ization
0.0
mator (mean shift), constant and linear. Sup- pose that x and correspond to samples from distributions p and q, respectively. Then for constant estimator the transport map is given by T (v) = [bg. For barycenter transla- tion estimator, the transport map T has the form T(x) (v) (x)) = x + Mg Mp Finally, the linear estimator is given by:
100 of generalizatio
9000-009
1.5
2.0
CT solver
OT solver
Moan shit:
Mean
Constant
3.5
10g1: of N.
Ingio of N.M
Figure 5: Estimation errors of the baseline solvers using a limited number of available empirical training samples M.
T(x)= Σ (
Ep (x pp) Mg.
We refer to (Korotin et al.) 2021b) for the additional details on these baselines. We run the solvers in the experimental setup of $5.1 using the limited number of available training samples N. M for learning the means and covariances of input and target distributions. The estimation errors are reported in Fig. 5
The results show that the linear estimator achieves the strongest performance among all baselines. Still, the estimation errors of the OT solver significantly lower than the errors of all baseline solvers. Besides, as expected, these non-neural estimators do not show noticeable dependence on the number of available training samples.
31

---

Published as a conference paper at ICLR 2026
DIM-
2.5
20-
Sape
Expetiranta
Sippe
Eepet costmartalon
Dxpertarnal Sipe
Goorzo cnabteeb Racrapii Sappe -2.003
collede eacraade cileeone Plaue -c.)
Pritooo sovil noopocn.....
conntoo Butee
50
0.5
Skipe
-1.0
N,M
NMM
N.M
NLM
NM
Figure 6: Convergence rates of the OT solver learned with the quadratic transport cost and a limited number of empirical training samples for distributions of dimensions D 8, 16, 32, 64, 128.
Figure 7: Visualization of the solutions of OT solver under the experimental setup from $5.
DIM 2, max Hf 8. The figure shows that the usage of shallow architectures for potential H leads to improper collapsing solutions.
C.2 HIGH-DIMENSIONAL EXPERIMENTS
In this section, we report the estimation error of OT solver in dimensions D = 8, 16, 32, 64, 128.
We follow the experimental setup described a5. visualize the results Fig. 6 The results confirm our main conclusion that the estimation error of the minimax OT solver decreases with the increase of the number of empirical training samples N, . w.r.t. log scale.
Meanwhile, we plot the slope of convergence of the logarithm of the eтror w.r.t. the logarithms of N, and observe that it is not always less than -0.5. Besides, we do not observe any evident dependence between the dimension and the slope of convergence, e.g., the highest dimension D 128 corresponds to the convergence with the steepest slope < -0.58. These observations could be explained by two main reasons. First, Corollary 4.10 establishes the upper bound the estimation error. Thus, while upper bound theoretically exhibits linear convergence with the slope equal to 0.5, it does not guarantee that the error itself exhibits the same rate of convergence. Second, as was noted in $5.1 the theoretical bounds might be corrupted by the optimization errors. These optimization errors might have different effects on the generalization error depending the number of available training samples N, M; analysis of their roots and consequences represent an interesting future research direction.
C.3 APPROXIMATION ERRORS FOR SHALLOW ARCHITECTURES
In our experiments illustrating the effects of the approximation error ($5.2). we discussed the roots of potential improper solutions and large approximation errors in the case of extremely shallow architectures for potential He and/or transport map Tw- For example, in our Fig. 4 (main text), one might notice that the approximation error is accumulating at specific points, see the values for max H. = 4, (D max Hs 8) or max Hy = 1,2 (D 4, max H. 4).
To further clarify the results of the OT solver for the case of extremely shallow architectures, we visualize solutions in Fig. 7 Specifically, we plot the results of OT solver trained in the experimental setup described in $5. for dimension D 2 and the transport map T. with the maximum width of the hidden layer equal to 8. According to the approximation error reported for this experiment in Fig. the solutions are divided into two modes the mode with higher error corresponds to extremely shallow potential architectures, s.t., max H. = 4, 16. The mode with smaller error corresponds
experiments with wider architectures of potential yo, max H, 32, 64. Our Fig. 7 shows that the results of the OT solver with shallow architectures are indeed collapsing and represent bad approximations of the target samples which could be explained by the
32

---

Published as a conference paper at ICLR 2026
low expressivity of the used nets, At the same time, wider nets provide fine approximations of the target samples and are slightly different to each other.
D EXPERIMENTAL DETAILS
General remark. We parametrize the transport map Tw using MLP architecture with ReLU activa- tions. For the potential ço, we use ICNN architecture with the quadratic skip connections. Note that in our theoretical investigations, we consider more traditional ICNNs with linear skip connections, see $ However, we decided to consider another type of ICNN for potential parametrization in order to minimize the impact of the approximation error since these ICNNS were used to construct the benchmark pairs from (Korotin et al. 20215). While we can not totally eliminate the approximation error coming from the parametrization of OT map Tw. we strive to decrease it by selecting rich NN architecture (x= 2 mln of parameters). Importantly, ICNNS found their popularity in a number of specific (e.g., OT) and general purposs generative models, see (Huang et al. 2021 Dugan et al.
2023 Kornilov et al., 2024
Amos 2023: Mokrov et al. 202T).
Other details. Each experiment requires several hours of training on one A100 GPU. We use Adam optimizer (Kingma & Ba
2014) with Ir=le-3. In $5.1 we use batch size equal to min(1024, N) for run the solver for 10K steps, performing 10 inner iterations of updating weights of Th at each step. We calculate the error between the learned f and true T OT maps: II T [I= using 21 test samples. We implement standard linear regression using numpy. polyfe yfit package. For each experiment in and Appendix C.2 we run it times with different random seeds, log the errors and report their mean.
each dataset size N = M. $5.2 we use batch size equal to 1024.
E GENERATIVE ADVERSARIAL NETWORKS: RELATION TO THE SEMI-DUAL OT SOLVERS & THEORETICAL ANALYSIS
The objective of the semi-dual minimax continuous OT solvers resembles that of GANs (Goodfel. low et al., 2014). This fact motivates us to discuss the relation of the OT solvers to GANs and review some of the existing theoretical results on GANs below.
E RELATION OF SEMI-DUAL MINIMAX CONTINUOUS OT SOLVERS TO GANS
While there are many papers on GANs (Pan et al. 2019) and relatively large number of works studying the theoretical aspects of their objectives, they have limited relevance to our study for several reasons. First, OT solvers and GANS pursue different goals. The main goal of OT solvers is to approximate true OT maps 7 i.e., to learn particular optimal mapping (generator) between the given source and target distributions; accordingly, our theoretical work focuses on the error of this approximation. Meanwhile, GANs are focused on learning an arbitrary mapping which pushforwards some noise to a target distribution; thus, existing theoretical results on GANs mostly investigate the error of approximating the ground-truth distribution q by the generated Pgen- Second, OT solvers and GANs have an evident difference corresponding to the order of optimization over generator G' (map T') and discriminator 1) (potential (o) in their objectives. In particular, for GANs, the optimization over the generator G is done in the outer problem of their objective, while for OT solvers, the optimization over OT maps is conducted in the inner problem, i.e., the OT map T corresponds to the solution of the inner problem. We emphasize the importance of this difference for OT solvers, OT map T corresponds to the solution of the inner problem, which makes the task of theoretical investigation of the error of T* approximation even more difficult.
Besides, we outline the theoretical and methodological differences between GANs and semi-dual OT.
The origin of GANs training objective. We have some noise Pz, (parametric) generator model G and target distribution q, and want G = g. To achieve this goal, we use a discrepancy, D( q). which compares generated and target distributions. Theoretically, the discrepancy could be anything, e.g., KL (JSD), MMD, Wasserstein distance, etc. The discrepancy ensures the generated and target distributions maps p= to q would result in zero discrepancy). Also, the order of practical optimization is also dictated by the explained scheme. Our aim is to solve ming D( q), i.e., the outer optimization
close, but does not restrict G to be of a particular form (i.e., any G which
33

---

Published as a conference paper at ICLR 2026
is w.r.t. G. The inner adversarial maximization, in turn, appears due to particular variational form of the discrepancy D.
The origin of semi-dual OT stems from objective eq. in paper. Lc.. there is known theoretical result that optimizing eq. ( w.r.t potentials & results in OT cost (therefore, the outer optimization of semi-dual OT is w.r.t. potential (or diseriminator), not mapping (generator)). The other theoretical result states that Optimal Transport mapping between and is (x) where o the solution to eq. (3). Therefore, the final objective we study in our work, eq. (6), is almost the same as eq. 3 but with Fenchel conjugate appearing in (3) substituted with the auxiliary learned argument of Fenchel transform, which is known be exactly Vi.
As we can see, the origins of semi-dual OT and GAN-based training drastically differ, making the distinction clearer.
F.2 THEORETICAL ANALYSIS OF ADVERSARIAL. GENERATIVE MODELS
Below discuss two groups of theoretical results on GANs which are slightly relevant to
Estimation error of distance between generated and true distributions. The work of (Arora сt al. 2017) introduces a notion of F-distance between the pair of measures w.r.t given class of discriminators F:
dr 4) sup En-p + Eyng - D(w)) - (
DEF
Here vb [] + R is any monotone concave function. For certain classes of functions, these distances define an inner optimization problem for well-known GANs, e.g., Wasserstein, Vanilla, MMD, Sobolev, etc. Class Fun corresponding to neural networks with a bound on a number of parameters yields so-called neural nel (NN) distance. The paper (Arora et al. (017) establishes an upper bound on the estimation error of this distance, i.e., difference of NN distances dr..,. (g, Pgen) for empirical and continuous distributions. Subsequent work (Ji & Liang 2018) extends this analysis by providing tighter upper bound and lower bound for this difference. in some sense, these works analyze the error in GANs inner optimization problem coming from the usage of empirical measures p. instead of the real ones. Still, these works ignore the errors rooted in outer optimization problem of GANs.
Estimation error of GANs. Other class of papers takes into account errors in GANs' outer opti- mization problem. They commonly consider the solutions of GANs' empirical optimization problem: @ arg min d=.., (q. Pgen(w)) where Pgen(w) denotes the distribution generated from empir- ical samples of distribution p by a generator G (with parameters w from the class f2). Then the papers derive upper bounds for the estimation error defined as
inf dz(4, Pgen(w)) mEE
for different variants of function classes F. For example, (Ji al. 2021 derives an upper and minimax lower bound for this error and Fmn class; (Zhang et al. 2018) considers the same class but a different type of error using w² arg min d Poen where Pgen(w) the distribution generated by G. from (Liang 2017) and (Singh et al. 2018) study the same type of error but consider the discriminator class Sobolev space; (Uppal et al., 2019) extends these result by modeling the discriminator and generator as Besov function classes. In general, these bounds are constructed based on Rademacher complexities of function spaces and are utilized to construct the bounds on the distance between Pgen and g.
For completeness, we mention several other works which investigate theoretical propertics of specific GANs, e.g., Vanilla (Biau et al. 2020), bidirectional (Liu al. 2021). Wasserstein (Biau et al. 2021
and CycleGAN (Chakrabarty & Das 2022), (Sun et al. 2024). (Moriakov et al. 2020).
F LLM USAGE
LLMs have been used only for polishing of the text purposes, i.e., for checking grammar and spelling, and for assistance with sentences rephrasing.
34


Reviews:
[{"review_id": "mu4jtWkgNw", "reviewer": "Reviewer_umCk", "summary": "The paper develops a statistical learning framework for the proposed method, stating finite sample guarantees and excess risk bounds for the estimator. It highlights how regularity and beta-strong convexity assumptions drive stability and convergence rates.\nThe analysis connects optimization error and estimation error, with explicit dependence on sample size and ambient dimension. The results are positioned relative to statistical optimal transport, where dimension often controls rates.", "questions": "1. Under what concrete distributional conditions on the source and target does beta strong convexity hold? \n2. The estimation error exhibits explicit dimensional dependence that resembles known lower bounds in statistical optimal transport. Can the authors clarify whether structures such as low intrinsic dimension, manifold support, or spectral decay can reduce the exponent, and if so, how this would change the bounds and constants?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Theoretical framing is clear, separating approximation, optimization, and estimation errors. The proofs use standard tools from empirical process theory and stability.", "weaknesses": "The beta strong convexity condition is assumed rather than derived from the model or data-generating process. Please either prove it under verifiable conditions or provide a constructive check that practitioners can apply to certify it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pLcQzgw5tE", "reviewer": "Reviewer_BuZV", "summary": "The paper studies the approximation capabilities of a certain class of optimal transport problems for quadratic cost.  In particular, it focuses on the estimation of the *generalization* error: this stem from several factors, the approximation error (due to the dual potential being parametrized with a neural network) and the estimation error (stemming from finite sample).\n\nThis family of problems (and their solutions) is well studied, but the case of semi-dual solvers is less studied. Here, the semi-dual formulation allows to parametrize a potential using a convex class of functions, the ICNN. \n\nThe experimental part of the paper study a particular implementation of the approach using Input Convex Neural Network (ICNN).\n\nI did not check the proofs in appendix", "questions": "### Approximation power as function of depth\n\n**In Figure 4, Dim 2**, I spot a counter-intuitive behaviors. I see that the approximation error does not diminish with $\\max H_{\\phi}$. Do you know why? This somewhat contradicts Theorem 4.3.  \n\nCan you clarify what is the **x-axis**: why does it cover the 0-1 range? \n\nFinally, we sometimes see the error accumulating on special values (e.g Dim 2 maximum width, or Dim 4 small width). Can you double check that the network is not degenerating toward a trivial solution?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper is well written, very clear, and accessible to people with familiarity in optimal transportation, learning theory and statistical consistency. Overall, the strategy to tackle the problem is well explained.  \n\nThe related work is well done, especially on semi-dual formulations, which are a getting popular, making the contribution timely.  \n\nThe methodology is standard, but the result is new. Only a subset of results actually depend on the approximation property of certain neural networks classes. Others are a classical decomposition of the error between generalization and approximation.\n\nWe can appreciate the diversity of tools and techniques used.", "weaknesses": "### Missing related work\n\nThe recent work of Nietert and Goldfeld is relevant and could be added to the related work.\n\nNietert, S. and Goldfeld, Z., *Estimation of Stochastic Optimal Transport Maps*. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.\n\n### Somewhat unconvincing experiments\n\nMy enthusiasm for the paper is somewhat tempered by the numerical results. \n\n**In Figure 3**, the logarithmic scale hides many phenomena. For example, I'd be curious to see what the experimental error looks like with the naive barycenter translation estimator: $T(x)=x+(\\mathbb{E}_Q[x]-\\mathbb{E}_P[x])$.\n\nI have similar concerns with Figure 4 (see questions).  \n\nOverall, since some experiments do not depend on neural networks, and since ICNN are notably hard to train, it could be good to plot results of other (possibly non-neural) parametrizations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pNSVbbSWoL", "reviewer": "Reviewer_aRUC", "summary": "The paper provides a theoretical characterization of the generalization error for optimal transport mappings parameterized by input-convex neural networks, focusing on a class of continuous Wasserstein-2 distance solvers. The analysis decomposes the overall error into two components, estimation error and approximation error, each of which is independently derived and subsequently combined to yield the final generalization bound. The theoretical findings are supported by experimental validation on a simple, low-dimensional synthetic dataset, which confirm the consistency of the results with the proposed theoretical framework.", "questions": "In Figure 3, the slope appears to increase as the dimensionality rises. Could the authors provide further insights into the underlying reasons for this behavior? Additionally, as the dimensionality continues to grow, do the authors expect the slope to remain below or equal to 0.5?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper makes an original theoretical contribution by deriving generalization error bounds for quadratic optimal transport (OT) solvers parameterized by neural networks, particularly within the class of input-convex models. This work provides valuable insights into the learnability and error behavior of neural OT mappings and offers practical guidance for their use in learning-based transport problems. The theoretical framework is rigorous, decomposing the overall generalization error into estimation and approximation components with clear mathematical justification.\n\nThe paper is clearly written, logically structured, and easy to follow, even when presenting technically complex material. Although the experiments are limited to simple synthetic datasets, they effectively validate the theoretical results. The significance of this work lies in establishing foundational learnability guarantees for neural OT solvers, which bridge the gap between theoretical understanding and practical implementation in modern OT-based learning frameworks.", "weaknesses": "A key limitation of the paper is its exclusive reliance on low-dimensional synthetic datasets for empirical validation. While the theoretical analysis is sound, the absence of experiments on higher-dimensional or real-world datasets limits the assessment of the framework’s practical relevance and robustness. Extending the experiments to more complex domains would provide stronger empirical support for the theoretical claims and demonstrate the scalability of the proposed bounds.\n\nAdditionally, the paper assumes \\beta-strong convexity, a condition that is often difficult to guarantee in neural network parameterizations. While the authors acknowledge this assumption as restrictive, a deeper discussion of its implications and potential violations would be valuable.\n\nSome missing references, e.g. [proof ref.] in multiple theorems and propositions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5srI3gN77W", "reviewer": "Reviewer_1LsA", "summary": "The authors study a particular semi-dual formulation of the quadratic Optimal Transport (OT) problem and derive statistical guarantee bounds for recovering the true OT map when using a solver that optimizes this formulation. More precisely, they adopt a generative modeling framework and assume that the input and target distributions are absolutely continuous probability measures given through samples. They consider the case where the solver relies on two neural networks optimized using the semi-dual objective, which is approximated via Monte Carlo estimation based on the empirical distributions of the measures. The authors provide an upper bound on the L2 distance between the true OT map and the best transport map that could be obtained by the considered solver. To achieve this, they first decompose this gap, referred to as the generalization error in the paper, into two terms: the estimation error, which accounts for the discretization of the semi-dual formulation, and the approximation error, which arises because the optimization is performed over a restricted class of neural networks rather than the entire space of functions. They bound the first term using the Rademacher complexity of the function class and show that the second term can be made arbitrarily small by choosing an appropriate class of neural networks.", "questions": "- Does the class of neural networks described in Proposition 4.4 represent any $\\beta$-strongly convex function that is totally bounded with respect to the Lipschitz norm?\n\n- What does assuming that the optimal potential $\\phi^*$ is $\\beta$-strongly convex imply about the considered distributions $p$ and $q$ ? \n\n- In Section 5.1, you assume that, because you use the same neural network architecture as the one corresponding to the ground-truth OT map, there is no gap in the approximation error, and that the observed generalization error is, in fact, the estimation error. This seems to be a strong assumption, as it implies that the optimization process reaches the global minimum of the loss defined in Equation (8) and does not get trapped in local minima when optimizing the ICNN (a similar comment applies to Section 5.2). Could you comment on this? Do you observe that, for different initializations of the ICNN, the optimization consistently recovers approximately the same transport map? Or, did you try initializing the neural network with several random seeds and retaining the best result?\n\n- Your experiment in Section 5.1 seems ideal, as it suggests that the bound you derive is tight in dimension 4—is that correct? I am not a specialist in statistical bounds, but is it common to verify experimentally that such bounds are tight as you did? Could you elaborate on this?\n\n- During the experiments, did you constrain the ICNN as described in Proposition 4.4? Do you observe that imposing these constraints helps the optimization in practice?\n\n- Finally, there is a small typo: it should be $\\varepsilon / \\beta$ in Equation (20), although this does not affect the result.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The theoretical results presented in this paper are interesting: the authors derive an upper bound on the number of samples required to approximate the true OT map using the proposed solver. Section 4, which focuses on deriving this upper bound, is well organized and clearly written, which permits to easily follow the main steps of the derivation.", "weaknesses": "- I am a bit perplexed by the experimental setup. In Section 5.1, the authors assume that, because they use the same neural network architecture as the one corresponding to the ground-truth OT map, there is no gap in the approximation error, and thus the observed generalization error corresponds solely to the estimation error. This appears to be a strong assumption, as it implies that the optimization process finds the global minimum of the loss defined in Equation (8) and does not get trapped in local minima when training the ICNN.\n\n- Regarding the writing, Sections 2 and 3 could be improved. The theoretical background on OT with general cost functions is not necessary to understand the paper’s contributions. The authors should focus exclusively on the quadratic cost in Section 2. Similarly, the paragraph “Continuous OT solvers for quadratic cost” does not seem essential for understanding the main contributions.\n\n- Finally, the writing should be polished and carefully revised for clarity and precision (see, for example, lines 352–353, 405–406, and 410–411).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)