Mean-Field Neural Differential Equations: A Game-Theoretic Approach to Sequence Prediction
Authors: Sungwoo Park
Byungseung Kong
Venue: ICLR 2026 Poster (2026)
Keywords: Mean-field games
Differentiable Games
fictitious play
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: We propose a novel class of neural differential equation models called mean-field continuous sequence predictors (MFPs) for efficiently generating continuous sequences with potentially infinite-order complexity. To address complex inductive biases in time-series data, we employ mean-field dynamics structured through carefully designed graphons. By reframing continuous sequence prediction as mean-field games, we utilize a fictitious play strategy integrated with gradient-descent techniques. This approach exploits the stochastic maximum principle to determine the Nash equilibrium of the system. Both empirical evidence and theoretical analysis highlight the unique advantages of our framework, where a collective of continuous predictors achieves highly accurate predictions and consistently outperforms benchmark prior works.

Published as a conference paper at ICLR 2026
MEAN-FIELD NEURAL DIFFERENTIAL EQUATIONS: A GAME-THEORETIC SEQUENCE PREDICTION
Sungwoo Park Byungseung Kong
Korea University [sungwoc_park, xncb135) @wwrea. krr
ABSTRACT
We propose novel class of neural differential equation models called mean- field continuous sequence predictors (MFPs) for efficiently generating continuous sequences with potentially infinite-order complexity. To address complex inductive biases in time-series data, we employ mean-field dynamics structured through carefully designed graphons. By reframing continuous sequence prediction mean-field games, we utilize a fictitious play strategy integrated with gradient- descent techniques. This approach exploits the stochastic maximum principle to determine the Nash equilibrium of the system. Both empirical evidence and theoretical analysis highlight the unique advantages of our framework, where a collective of continuous predictors achieves highly accurate predictions and consistently outperforms benchmark prior works.
INTRODUCTION
Modeling spatiotemporal processes central to understanding and predicting the behavior of complex systems that evolve across time and space. Recent work neural differential equation models (Chen) (et al. 2019 Tzen & Raginsky 2019) has shown that such architectures can effectively capture spatiotemporal dynamics in a wide range of applications, including generative modeling (Song et al. 202T), quantitative finance (Cohen et al. 2023), and physics- -nfficeed learning (Iakovlev et al. 2024). However, most of these approaches are formulated and evaluated under fixed, finitely sampled time grids, and therefore provide only limited theoretical insight into the following question about inherently continuous sequences: How can we systematically model continuous the temporal discretization is refined and the effective number of events becomes very large? In this work, we address this question by directly formulating the data dynamics continuous time and then studying their behavior in the regime of increasingly fine temporal granularity. To obtain a tractable and theoretically grounded framework, we cast the prediction problem into the setting of mean-field games (Lasry & Lions 2007), which gives rise to an infinite-dimee dimensional predictive decision decision-making model that extends existing neural differential equation approaches (Tzen & Raginsky 2019) to the analysis of continuous-time sequences.
sequences as
The mean-field principle, a core philosophy in various scientific domains including neuro- science (Faugeras et al. 2009), statistical physics (Negele) 1982). and economics (Carmona) 2020 Cardaliaguer & Lehalle 2018), serves as a powerful tool to model and analyze a large number of interacting agents, who behave in manner that can be described tragically rational within the decentralized coalition to satisfy Nash equilibrium. In this state of the mean-field regime, continuum of infinitely many agents individually governs the dynamics of partially observed historical sequential data and collectively interacts with the others to make optimal group decisions for the prediction of future events. The foundational principle of this game-theoretie interpretation of the predictive system can be stated as follows: We reconstruct the continuous-timg sequence prediction problem under the formal lens of mean-field games gain powerful generalization capabilities in continuous sequence modeling. Stemming from the principle, we offer two main contributions:
We extend conventional neural differential equation models by introducing mean-field principles, providing approach for modeling continuous sequences. This framework represents the stochastic spatiotemporal dynamics of an infinite continuum of agents and is rooted in hypotheses
"Corresponding author.

---

Published as a conference paper at ICLR 2026
f
Aggregation
p(u)
U2
fftut X
u3
Samping
Real Obsenvation
Yu
11
XTC
W (Grephoon)
Figure 1: (Left). The mean-field predictors are conditioned on set of labeled past observations {un p(u). Each spatiotemporal dynamic interconnected via the neural graphon W a. which leverages inductive biases tailored for continuous sequential data. (Right). The collective decisions a coalition of mean-field sandf predictors are calibrated to approximate (black trajectory) the target future event interval.
from time series analysis (e.g., seasonality). We demonstrate that our proposed method surpasses state-of-the-art benchmarks in continuous sequence prediction tasks including variants of state-space models.
We propose gradient-based mean-field FBSDE approach that provides feasible computational complexity for approximating Nash equilibria in mean-field games. Building on the concentration of empirical measures and the propagation of chaos property in the mean-field regime, our theoretical findings clarify the number past observations on the generalization performance of the mean-field system. We demonstrate that the coalition produces increasingly accurate and reliable predictions.
Problem Setup. Given past observations [yu) in the interval O C T = [0, T). the primary objective of the continuous sequence prediction task to accurately forecast future events {y:} within the interval t T T O. The continuous sequences {yury1} [0, [" + R2 are continuously defined and share three notable properties: (1) Irregularity. The temporal granularity between spatio-temporal states in sequential data varies. (2) Non-uniformity. The cardinality of sequences exhibits stochastic and non-uniform behavior, fluctuating each time it is sampled from the dataset. (3) Temporal scalability. The sequential data spans multiple time scales, encompassing both short-term fluctuations and extended temporal ranges.
2 MEAN-FIELD CONTINUOUS SEQUENCE PREDICTORS
This section starts by introducing stochastic differential equation model designed to depict infinite- order continuous signals, incorporating graphon structures for feature interactions.
Definition 2.1. (Mean-field Graphon SDEs) For the Markovian feedback fee controls a T x + Rdd (i.e., a xttxx; x; x))) and continuous labels p(u), we propose the Ra--ul controlled stochastic differential equations called mean-field gruphon dynamics defined as follows:
dx (t) = (t) bb) (X. (t), + b(t, X (t), at + t,dW!, (0) (1)
where a probability measure v == {v((0)(v))ExXT serves as a concise representation of the law of dynamics, and yu ~ y)u,y) denotes continuous representation of past observations.
The mean-field dynamics presented in Definition 2.1 involves three terms on the right-hand side, with an emphasis on important notions (A) mean-field predictors and (B) neural graphons.
(A) Mian-field Predictor. The proposed dynamical system incorporates two types of continuity encoding: locality (ie.,() 1) and labeling (i.e., u). The state variable X (1), termed continuum of predictors or mean-field predictors (MFPs), represent a continuous set of information flows, each labeled by p(u) and initialized from the past observation, (0) yu p(u,y). For instance, a continuum predictors for the sequence of infinite i.i.d labels U.o == {un ~ p(u); n < N + oo) in the mean-field regime X (0) be interpreted as being conditioned interval, i.e., the support the label distribution p(u), with their future causal outcomes, producing X (t) at future event interval being obtained from the dynamics in Eq (T).
the past observational
The suggested model effectively handles continuous signals by ensuring both input and output are processed continuously. Within this setting, the closed Markovian control process a<::; 0) E A,
ii

---

Published as a conference paper at ICLR 2026
parameterized by neural networks 8 E θ, neural agents, controls the state dynamies X (t). Fig [ depicts illustrative examples of how the proposed mean-field predictors are conditioned (left), propagated (mid), and utilized to produce future prediction (right). The overarching goal is then to calibrate the trajectory of predictors by determining the optimal neural agent a' that closely approximates the target future interval, e.g., Ee Bu Xu (t) - yell x O, where decision aggregation O [0, 1] captures the collective behavior of mean--iell predictors. Section will present a systematic algorithm to fulfill this objective.
(B) Neural Graphon. It is widely recognized in the literature that fundamental assumptions of inductive biases, such as temporal decay, cycles, and seasonality are vital for effective time series modeling. To incorporate these mean-field system, introduce neural graphon, graphon structure parameterized with neural networks, capturing the inherent heterogeneity among predictors.
Definition 2.2. (Neural Graphon) A graphon is a set of symmetric integrable function W : 02 - IR equipped with L.2 -orm. For a probability measure jt defined on x Rdd with bounded second moment, we define measure-valued function W Waluu(() O - *Maan and a continuous symmetric function 4b = sb x, x)) == Hap Pro))y - x) such that the first term in right-hand side of Eq (T is defined as (Wa (u), & (y, ax) E (yxpp() xopp (Wa v)sb (y,x)] E Rd.
"Please refer to Section A.2 for the deatils.
For two tuples (x, u) Mn & p(u) and (y, v) ~ n & p(v), a symmetric function db estimates scaled relative dissim- ilarity between spatial features and y. The neural agent, i.e., H. (a),, then adjusts the importance of dissimilarity by rescaling projected vectors, i.e., Proj. Meanwhile, the neural graphon W encodes degree of interaction between temporal variables and v. Among the various graphon designs available, propose two structures informed by inductive biases specific to continuous time series. Note
Exponential Graphon
Cosinusoidal Graphon
Figure 2: Visualization of Graphons. that the key distinction from conventional methods is that our approach directly models inductive biases in the data space Rd, rather than in latent feature spaces, facilitated by the graphon structure.
Exponential Graphon. In the first graphon structure, incorporate temporal decay (Che al.) 2018) assumption on spatiotemporal variables, which suggests that the influence of the past event decreases exponentially as time deviations increase. An example depicted in Fig 2 illustrates exponential graphon, highlighting that events occurring close in time often show strong interactions. Here, the function i.e., W1 A g+ determines the interaction magnitude. Subsequently, we introduce an exponential graph that diminishes the influence of events that are temporally distant; Wa (u, v) (a()) exp( T Au) with A. == | v|v.
Cosinusoidal Graphon. The second graphon is intended to highlight the continuous eyclic as- sumption (Oreshkin et al.J (020),, which characterizes the periodic aspects of time-series data. To embody this assumption, we conduct digen-decomposition of the proposed graphon operator on (O), employing sinusoidal eigen-functions (i.e.. (w7)) and different frequency modes for the eigenvalues (i.e., ), as suggested by Gao & Caines (2019); W = Id + Σx,EEz, Arpis where
{ C [Id, V2 cos 2 2 sin k and {Ar} C {ao. by 22. . We utilize neural networks to pa- rameterize the graphon operator, substituting the Fourier coefficients {Id, A1 with equivalent neural networks, specifically Wo. WT.1. W21 A R+ To illustrate different periodicities, we introduce f(I) E (1/2, 1/8)15L. which denotes pre-determined series of frequencies. Consequently, we define a cosinusoidal graphon as follows:
Wa (a v) Wo(a)
Σ (iu((a)cos( + W2,((a) sin(-). IE(1.....)
(2)
where (-)= 2mT(1) Note that we limit the summation finite modes (i.e., L) for computa- tional tractability. illustrats periodic interaction magnitudes for a predefined frequency setup. Further details the implementation and their analysis can be found in the Appendix.
iii

---

Published as a conference paper at ICLR 2026
3 TRAINING MEAN-FIELD NEURAL NETWORKS
In the prior section, we introduced mean-field continuous sequence predictors based on SDEs that incorporate spatio-temporal interactions. Given that the mean-fiche system in Eq (2.1) is characterized as controlled SDEs with neural agents, can define the objective function as stochastic control problem. More precisely, our main aim is to reduce the cost functional J formulated for training neural agents for sequence prediction and to derive the value function V:
Definition 3.1. (Cost functional @ For the given neural graphon War and fixed set of admissible controls A. the cost functional is defined as follows:
V inf (V inf Ba,,, Eupppu) X (I) -elli: Go] aEA
(3)
aEE
where G = G(X v*) represents the terminal cost at time t = T,, and w 0 - [0, 10.11
decision aggregation function, satisfying vu(uud (o(u)du-
"Please refer to Section A.4 for the details on the definition.
To predict future values, mean-field predictors operate by generating unified measure, specifically referred to as a temporal marginal of predictors Eu (t)........)))))( Here, the expectation accounts for the label u by amalgamating weighted outputs (i.e., w) from a range predictors u ~ p (u) =
w# [Unif (O) aiming approximate the target continuous interval {y, her. Figure 1 (right) presents a demonstration of the decision-making mechanism. In pursuit of producing target intervals, neural agents are conditioned to extract the value function V, which describes the state where continuum of players unites to collaboratively forecast the most favorable future occurrences.
The difficulty in addressing this problem arises because the neural agent affects the population of predictors oa which, in turn, persistently alters the individual state variables through interactions facilitated by the neural graphon. In the literature, these types of problems are typically described as (graphon) mean-field games (Lasry & Lions 2007| Caines & Huang 2021). In this study, we propose novel methodology to cast the continuous sequence prediction problem through the lens of mean-field games. Our main aim is subsequently to identify the most suitable optimal control ax that fosters the optimal response in the recursive interaction between V and va, In our analysis, we explore the derivation of exact solutions (V, yo ) from optimal control profiles over time, by studying the subsequent system of PDEs within the mean-field regime:
Definition 3.2. (Forward- Backward PDE System). For the obtained optimal neural agent a", exact solutions of the value function Eq 3 can be obtained by solving the following system of PDEs:
O.V(t,x) + /1, x) H(t, x, 0.V, v (t), (0)=)aa)))))) 0,
0. (1) (1)-f//206) (1)+ (1)<v([( (1), (1) + b(1,xxxa)) vi (1)(1) (1) -0,
where A and V. denotes Laplacian and divergence operators, respectively. The stochastic Hamilto- nian system H is given by
H(t, Xn, a, v, ax) (bw (Xm, v, a) + b(t, Xy, a)) a |EBu-p(jX_-- 1.11
where bw (x, a, = (Wal [w)), (x, a) is the graphon interaction term in Definition 2.2
system of decoupled PDEs consists of the Hamitton-Jacoke Bellmmm (HJB) equation and the Folker-Planck, Klookooooonnkko (FPK) equation, which individually describes the propagation rules of the state variable and the value function over time. In mean-fiet field equilibrium states, set of PDEs are coupled the law of the state variables Law X; (t)) matches (t) with marginal errors. This specific mathematical constraint can be formally expressed in the following definition:
Here, faj denotes a fush-forward prohability measure of i through function
iv

---

Published as a conference paper at ICLR 2026
Iteration
Iteration
Iteration
am-l-y +
(A) Information Propagation
(8) Update Control Profiles
v am
[Yulueo
L, P
Byast (7 VoY, -
(0]
4 (Projector)
V (Uplater)
Figure Illustrative Algorithm the Gradient System of FBSDEs.
Definition 3.3. (Mean-field e-Nash Equilibrium). We say that continuous flow of measure (-) an E-equilibrium of (graphon) mean-field games if there exists numerical constant e 0 such that the following inequality holds: sup,, (v,,t,, Law XXa (t)) 3 O(E), where a E l is optimal control of the problem in Eq.
The mean-field equilibrium described in Definition 3.3 characterizes a scenario where a continuum of predictors is not incentivized to modify their policies a' to non-optimal counterpart B, which induces marginal errors, ie., [v², , (3) Z J(va "a"). Here, the law of optimal mean-field pre- dictors closely approximates the population i, with marginal errors 6. This coupling integrates (FPK) equations, forming master equation. Several numerical methods exist to approximate solutions to mean-field games including fixed-point iterations (Lauriere) 202T), and fictitious play (Min & Hu 202I). However, these methodologies are typically constrained to linear quadratic dynamics, leading to computational intractability when confronting non-linearity (e.g., neural networks). Additionally, numerical simula- tions for obtaining analytic solutions of this system of PDEs present significant challenges due to the curse of dimensionality in high-dimensional data spaces. The following section is dedicated to addressing these issues by leveraging the deep neural architecture.
the Hamilton-Jacobi-Bell Bellman (HJB) and Biaieer-Planc--oonnnee
3.1 GRADIENT SYSTEM OF NEURAL FBSDES
Inspired by computational algorithms designed for fictitious play (Cardaliaguet & Hadikhanloo 2017). we explore a gradient descent-based algorithm, which enables us tackle solving MFGs by fusing deep neural architectures. To be more specific, we propose a gradient system of forward- backward stochastic differential equations (Bensoussan et al. 2013). which is adapted for reflecting the update of neural agents with respect to the gradient descent algorithm.
Definition 3.4. (Gradient System of FBSDEs For the fixed flow of measures v,,()) T T - P2 and the fixed label u at each stage m, we consider a family of processes (Xu(t), Y (t),), Zu(t)) that solves borward-backwar stochastic differential equations with respect to the proposed graphon system in Eq (1) given as follows:
dX.. mmmm (t) bi 6"d dt + o,dW?, d (t) - dt Z -dW
4m = Or (t,, X ama 0Th EY V. (t))), Vn = Law(X. Lo
where 7mh > O is a learning rate of the gradient descent at m-th stage, and {am C A set of admissible neural agents. Then, the triplet can be identified with (Y Y Yu())), Zt)))) = (J,G, (0xJ)o )
"For the detailed description of the FBSDE system, please refer to the Definition A.2
The proposed gradient system can be decomposed by iterating a two-step procedure, i.e., (A) and (B), over total of M stages. Fig illlustrates the evolution of the mean-field predictors related to the updated parameters of neural agents a'm across different stages m. The details of the two-step procedure are specified below.
(A) Information Propagation. Initially, the system disseminates the information to continuum of players by utilizing the population information of the previous stage, where the forward and backward system of SDEs propagates information relating to the updated population, V..
V/1
Law (X. m--.ai m. 1),
Y! Law (X"" (v,) @ Law (Y (Y""(/,)) (v.).
(4)
Note that the backward dynamics is propagated in reverse direction starting from its terminal state Y,, (T)) = G while the forward dynamics evolve in the forward direction from the initial state. This
shows that the proposed FBSDEs parallel the PDE system described in Definition 3.2

---

Published as a conference paper at ICLR 2026
(B) Update Control Profiles. In the subsequent step, the neural agent a is updated with respect to its parameter 9 following the steepest direction of minimizing the values of backward dynamics Y The backward dynamics, associated with the cost functional J described in Proposition A.22 guide the updates of the parameters, allowing the mean-field predictors to gradually approximate the target interval. Since we have proposed an iterative algorithm to solve MFGs, the remaining part aims to provide convergence guarantees and highlight optimality conditions.
guarantees that the gradient system in Definition A.2 induces optimal neural agents a', which yield a feasible value function (i.e., Y (0) m V) where the optimality of the control is represented in the sense of the Pontryagin stochastic maximum principle (Yong & Zhou 2012). Specifically, we have the following two results:
Proposition |
lim H (m)) = inf H a,), dt 03 dv, V ( J(vax ((x))
age
The result illuminates that pair olimm-+xo a 1 α², limm-+o am vo') solves both HJB and FPK equations in 3ef3.22 assuring stochastic optimality. Having obtained the value function, the next step is to provide an explicit estimation of margin in the convergence of mean-field equilibrium.
Convergence to Mean-field Equilibrium. To rigorously analyze the convergence to equilibrium in distributional sense, define two distinct operators. &b and v . + M,, referred to the projector and updater, respectively. Each operator corresponds to one of the two steps mentioned earlier, illustrated in Fig.
Φ v ) = (Law(Xū (1)) L TETTUEE),
(m-1)) = {van = an V = I JV -
can be easily verified that the composition of these operators at stage 771 maps the previous state's population to the next stage i.e., & V/ (-" = n" Proposition 3.5 asserts that the population (va") Img generated by the proposed algorithm begins to converge in the Wasserstein metric as the stages increase.
Proposition 3.5. (informal) For arbitrary u p(u) and i T, the m-fold of composition & w induces convergent behavior of squared 2- 2-Wasserstein distance:
W ( (v f a 1o)om (n))
supl Vo Y IE 0(7 C) = Em 17-
0. (6)
EE
where a numerical constant C is dependent on M, bo, C1, Has Lip, and m2, |0], e011, Lipw h(x) [Wall is a cut-norm of the proposed graphons (i.e., exponential, cosinusoidal)
Proposition 3.5 reveals two theoretical implications regarding the convergence property. First, the proposed gradient system converges in a distributional sense, the Wasserstein distance between the populations ([ (r)om (1))) 0m+1 and (( () (1))) o governed by the gradient increases. In other words, { v jo Cauchy sequence M, ensuring the convergent behavior the proposed training scheme. Second, the proposed gradient system ensures the convergence of the dynamics for the upper bounds Cm- It is important to note that the inequality in Eq (6) is an equivalent expression of the mean-field Nash -euuiiiibrium described in Definition 3.3 In this context, the neural agent with greater capacity (i.e., a smaller radius r.. of the metric balls in Eq (45) further tightens the upper bound. In conclusion, the findings from Proposition 3.5 validate that the proposed gradient system efficiently utilizes neural networks to solve mean-field games in continuous sequence prediction.
norm of the backward dynamics, is expected to decrease
SAMPLING MEAN-FIELD PREDICTORS
In this section, we propose the numerical algorithm for sampling the proposed mean-field predictors and provide a theoretical analysis the sample complexity eтror and the asymptotic convergence of empirical estimation for mean-field predictors. Inspired by the Fuler-Maruyana approach for Mckean- Vlasov type Reisinger & Stockinger (2022), we introduce an Euler- Maleer-Mruyamaa method tailored for ceaphon-interacting particle systems sample series of mean-field predictors.
Algorithm I in Section A.5 details the computational procedure for sampling these mean- meen-fieed predictors. We assume that a² == ax: 0*) is nearly optimal, defined by the e-Nash equilibrium
vi

---

Published as a conference paper at ICLR 2026
derived from the mean-field gradient system associated with FBSDEs. The dimenioad characteristics the proposed system introduce inherent complexity challenges when mean-fiet predictors sampled and applied to Aiite-dimensionne real-world datasets. When these sampled mean-field predictions aim to approximate their mean-field limits, an important question regarding sample complexity surfaces. To thoroughly investigate, we begin by defining the probabilistic description for both the sampled and model dynamics, as elaborated below:
1 N
MFPs in Alg. 1
Σ
(0): MFPs with co-order A == Bu-p(u) P (())-
where X. (1) vy is sampled predictors, which can be obtained from implementing the Algorithm and the weighted sum A, approximates true collective prediction made by mean-field predictors x ~ Rt in Eq (3)
Proposition 4.1. (Sampling Complexity) For arbitrary E O, let v. fūg be probability measures
defined in Eq ( Then, there exist numerical constants c4, C7, Cg, 0g > 0, w. > 0 and k. > such that the probability of squared 2- Wasserstein distance can be controlled as follows:
2c7 A =
sup P [W2 P.) 2 c] S ( + +
exp(cgedal1) (crt--) V co exxp(444c))
iET
e N22
Ne
1 128wh(a)
-d/8
B
a= 7
D
e
N
N
The established inequality presents the relation between squared 2- Wasserstein distance and the number of samples N, the dimensionality of the data distribution d. The proof primarily draws on the findings presented in Bolley al. (2007). It is important to note that the result also guarantees the proposed system benefits from the propagation of chaos (Chaintron & Diez 2022), validating the asymptotic behavior of the sampled predictions generated by the mean-field predictors:
sup lim W (Law .'...... X (11=1,)-)33///((()) s k) +
IET t-o
Proposition A.5 and the inequality above align with the intuition that as the number of predictors N increases (and dimensionality d), the sampled dynamics converges more closely the mean- field limit fúg and (t). In particular, the right-hand side of the inequality is influenced by two exponentially decaying terms, while the other term decreases short-tailed concentration with respect to the number of past observations. Overall, our theoretical findings highlight advantages of capitalizing on mean-field games: Rational individuals (i.e., (x) satisfying Nash equilibrium and conditioned on partial information (i.e., (.) = yi/n) forms a coalition (i.e., vVV), ), and the group decision is progressively refined to collaboratively solve the continuous sequence prediction problem. As the coalition size increases, the resulting predictions
polynomial rate, both showing
become progressively more precise and reliable. In Section 6 we conduct an ablation study to numerically verify these theoretical findings.
5 RELATED WORK
Neural Differential Equation Models. In recent years, neural differential equation models have gained attention for their ability to capture the dynamics of complex continuous sequences. Latent ODEs (Rubanova et al. 2019) extend standard RNNs to handle continuous signals by integrating neural ODEs with them. Kidger al. (2020) introduced differential equation models based on controlled differential equations (Neural CDE) to address key limitation of neural ODEs, where solutions depend solely on initial conditions and not on subsequent observations. Recently, Contif- ormer (Chen et al.] 2024) was developed, combining neural ODEs and Transformers into a single framework. Another line of research integrates stochasticity by utilizing SDEs, particularly for time-series applications. For instance, Latent SDE (Li et al. 2020) encodes sequential data in the latent space using neural SDEs, while MaSDE Park et al. (023) employs a concept of stochastic differential games to analyze time series. Koshizuka & Sato (2023) proposed a regularized neural SDE based on the Lagrangian Schrödinger bridge, and Oh et al. (2024) introduced three stable types (classes) of neural SDEs: Langevin-type SDE, Linear Noise SDE, and Geometric SDE.
vii

---

Published as a conference paper at ICLR 2026
45- 1e-2
2641
3e-3
1.30
4r5
V7yy [bp)
3.0
Varra
lest MStss
Test MALS
1:25
1.5-
41-
1.3
29
1.15
0.0
2
15
N1 of Sampling!
14 of Sampling)
Figure 4: Ablation studies on Figen Worm dataset. (Teftt) Sensitivity analysis on the sample complexity. (Right)
Robust analysis on post-intervention of non-informative signals.
MIT Humannid Robot
MIMIC-II
Heijing Air Quality MAE
Methods
MSE
MAE
MSE
MAE
MSE
Neural Luplace
8.11=0.25
17.03=0.33
7.7620.04
18. 18.70+0.00 08
3.21=0.12
11.45#0.23
MaSDEs
16.51=0.21
27.89:0.30
8.41=0.06
20.67+0.08
3,47=0.03
13.13:0.07
CRU
32.08=5.07
42.50+3.90
13.09+0.31
24.68+0.47
3,48=0.06
12.76+0.19
Latent SDE
6.01=0.14
15.94=0.14
8,04=0.02
19.63+0.06
3.29=0.03
11.99=0.07
Neural LSDE
6.80+0. 14
16.51
7.93=0.05
19.09
3.74=0.04
11.98+0.15
CONTIME
6.88+0.29
16.60=0.25
12.29=0.14
25.2640.
5.15=0.17
15,86+0.27
Contiforner
5,94+0.23
15.29=0.26
7.90=0.12 12
19.05+0. 18
3.25+0.10
11.48+0.16
$4
5.5910.11
13,9820.19
13,2410.01
24.79+0.30
3.9520.15
12.35:0.17
Mamha
5.21+0.09
13.71
13,2340.02
24.76+0. 19
3.6840.14
11,5643.24
Jamba
5.1340.13
13.32+0.20
9.7120.09
21.3740.06
4.0320.10
13.0440.20
MFPs (Exp.)
3,31=0030
10.12:0..2
7.51=0.08
18.59=0.11 18.6040
2.98=0.15
10.06+0.31
MFPs (Cosin.)
3.91
1143+0.07
7.51=0.06
3.1320.07
11.3820.08
Table Mcan Squared Errors (MSEs) and Mean Absolute Errors (MAEs) in various continuous sequence pre- diction tasks. The top and second-top scores in each dataset are highlighted bold and underlined, respectively. Each metric is scaled by 10
Mean-field Principles in Generative Models. Recent works utilized the mean-field principle to model the infinitely many random particles in high-dimensional data space, where they interact with each other. In Liu et al. (2022), the Schrödinger bridge was incorporated to address mean-field games in order to approximate data distributions for large populations. Park et al. (2024) introduced the concept of propagation of chaos to generate data structures with exchangeable high cardinality such as 3D point clouds.
EXPERIMENTAL RESULTS
Datasets. the experiments, we evaluate our results against benchmarks using the following datasets: (i) MIT Humanoid Robot (Li et al. 2024), (ii) MIMIC-II (Silva et al.] 2012), (ii)) Beijing Air Quality (Zhang et al. 2017), and (iv) EigenWorm [Bagnall et al. 2018). The MIT Humanoid Robot dataset contains the robot's state trajectories during various activities, such as running, jogging, and stepping in place, with 27 features describing these states. The MIMIC-II dataset, from the PhysioNet Challenge 2012, consists of time series data with 41 features representing the first 48 hours of a patient's 's ICU admission (e.g., SaO2 and cholesterol levels). The Beijing Air Quality dataset contains time series data for six air pollution indicators, collected from 12 different locations in Beijing. The Eigen Worm dataset comprises six features that characterize worm motion by projecting their shapes onto the six principal eigenworms, providing continuous sequences of 1500 length. To ensure consistent training, we apply either min-max and Z-score normalization on each data instance.
Benchmarks. Given our focus on continuous sequence modeling, the benchmark baselines consist of various continuous models, including Neural Laplace (Holt et al. 2022), MaSDEs (Park et al. 2023), CRU (Schirmer al. 2022), Latent SDE (Li et 2020). Neural LSDE OO al.) 2024). CONTIME (Jhin et al. 2024). and Contiformer (Chen et al. 2024). To further enhance the baselines, we also incorporate continuous state-space models, such as $4 (Gu al., 2022), Mamba (Gu & Dao 2024) and Jamba Lieber et al, (2024), Performance evaluation is carried out using mean squared error (MSE) and mean absolute error (MAE) metrics. Each model executed five times, with the average scores and standard deviations reported.
Quantitative Results. The main table presents a performance comparison with benchmark method- ologies across three datasets. The results show that the proposed MFPs consistently outperform other benchmarks by significant margins on all datasets. Notably, conventional neural differential equation models perform reasonably well on the MIMIC-II and BAQD datasets, where sequences
viii

---

Published as a conference paper at ICLR 2026
Ossenvation
Manbs
0.0
junda
-0.22
Ossorvation
-0.4
Mamba
MPPS (Exp.
Comiiommer
-0.5
-15
Figure 5: Qualitative results Prediction (Left) MIT Humanoid Robot datasct (Right) Eigen Worm datasets.
irregularly sampled with missing values. However, they exhibit performance drop on the MIT Hu- manoid Robot dataset, likely due to their limitations in handling complex spatio-temporal dynamics. In contrast, state-space models excel on the MIT Humanoid Robot dataset but experience decline in performance on the other two datasets, indicating their limitations in dealing with irregularly sampled sequences. Figure 4 (right) (right) illustrates the qualitative prediction results the MIT Humanoid Robot dataset. As shown, our MFPs deliver superior performance compared to the other models.
Long-term Prediction. The first study aims to demonstrate the ef- ficacy of utilizing mean-field principles for accurate long-term predictions. We com- pare the MFPs against variants of state- space models, recognized for their capa- bility in managing long-range dependen- cies Zhan et al. (2024) for long sequences. Table ?? along with Fig. A provide both quantitative and qualitative comparisons with benchmarks for long-term prediction tasks using the EigenWorm Worm dataset. Evidently, our MFPs achieve significant advances over other benchmarks.
Ablation Study I:
Eigen Worm
Methods
MSE
MAE
S4
14.16=0.18
28.69 33.69+0.22 21
Mamba
15.79=1.03
29.80=1.06
Jamba
17.63+1.09
31,96*1 .04
MFPs
12.52=0.16
26.61#0.29
Table 2: Long-term prediction on Eigen Worm.
Ablation Study II: Sample Complexity. To empirically validate the theoretical results derived in Section we conduct ablation study examining how predictive accuracy scales with the number of sampled predictors. As depicted in Fig. 4 (Left), our results align well with the concentration bounds formalized in Proposition 4.1 confirming that the empirical error diminishes as the coalition size of predictors increases. This behavior is consistent with the propagation-of-chass property of mean-field systems, whereby larger predictor ensembles more faithfully approximate the infinite-agent limit distribution. In practice, this translates to monotonic improvements in both MSE and MAE as N increases from 16, suggesting that additional predictors systematically enhance stability and reliability of forecasts. Nevertheless, these gains must be balanced against the computational overhead introduced during inference.
Noise Robustness. We perform robustness study to assess the impact of non-infonn con-informative noisy signal (i.e., white noise) interventions in past observations. Specifically, we inject the Gaussian random noises with variance Onaise = 0.3 to derive the distributional shift of test continuous- tiime sequences and corrupt the test data, p(u,y' y) = p(u, (Oa, Tuoise I2), where @ is convolution operation. Fig 4 (right) shows a uniform performance degradation (i.e., A) with an increasing number of past observations corrupted by non-informative noisy signals. As can be seen, our MFPs exhibit robust performance against noise interventions, as Mamba experiences sharp declines in accuracy under high levels of noise. Our MFPs under the coalition, trained on the original clean sequence p(u, y), neutralizes the influence of individuals conditioned on noisy signals p(u,y), thereby preserving the Nash equilibrium, resulting the robust generalization performance.
Ablation Study III:
7 CONCLUSION
This paper introduces mean-field continuous sequence predictors, novel class of neural SDE model for the efficient generation of continuous sequences, which can possess infinite-order complexity. We recast the time-series prediction problem as mean-field game and adopt fictitious play approach, integrated with a gradient- gradient-descent- based method, to leverage the stochastic maximum principle and identify the Nash equilibrium of the system. Both empirical and theoretical results reveal the distinctive features of our MFPs, where the coalition a continuum predictors generates accurate predictions and consistently surpasses benchmark performance.
ix

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was supported by ICT Creative Consilience Program through the Institute of Information & Communications Technology Planning & (Valuationn[ITTP) grant funded by the Korea govern- MMST( UTP-2206--832220--2211111

---

Published as a conference paper at ICLR 2026
REFERENCES
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018, 2018. URL /aatps org/ abs/1811. 00075
Erhan Bayraktar and Ruoyu Wu. Stationarity and uniform in time convergence for the graphon particle system. Stochastic Processes and their Applications, 150:532-568, 2022.
Erhan Bayraktar and Ruoyu Wu. Graphon particle system: Uniform-in-tinn inmm concentration bounds. Stochastic Processes and their Applications, 156(C), 2023.
Erhan Bayraktar, Suman Chakraborty, and Ruoyu Wu. Graphon mean-field systems. The Annals of Applied Probability,3333 33(5):388-33111 2023.
Alain Bensoussan, Jens Frehse, Phillip Yam, al. Mean field games and mean field type control theory, volume 101. Springer, 2013.
François Bolley, Arnaud Guillin, and Cédric Villani. Quantitative concentration inequalities for empirical measures on non-compact spaces. Probability Theory and Related Fields, 137:541-593. 2007.
François Bolley.
Quantitative concentration inequalities on sample path space for mean field interac- tion. ESAIM: Probability and Statistics, pp. 192-209, 2010.
Amarjit Budhiraja and Wai-Tong Louis Fan. Uniform in time interacting particle approximations for nonlinear equations of patlak-keller-segee type. 2017.
Peter E Caines and Minyi Huang. Graphon mean field games and their equations. SIAM Journal on Control and Optimization, (9(6))4373-4399) 2021.
Pierre Cardaliaguet and Saeed Hadikhanloo.
Learning in mean field games: the fictitious play. ESAIM: Control, Optimisation and Calculus of Variations, 23(2):569-59) 2017.
Pierre Cardaliaguet and GlaalessAAbert Lehalle. Mean field game of controls and an application to trade crowding. Mathematics and Financial Economics, 12:335-363, 2018.
Rene Carmona. Applications of mean field games in financial engineering and economic theory. arXiv preprint arXiv:2012.05237, 2020.
René Carmona and François Delarue. Probabilistic analysis of mean-field games. SIAM Journal on Control and Optimization, (1)):277552 2013.
René Carmona, François Delarue, et al. Probabilistic theory of mean field games with applications I-1I. Springer, 2018.
Louis-Pierre Chaintron and Antoine Diez. Propagation of chaos: A review of models, methods and applications. models and methods. Kinetic and Related Models, 15(6):855-005,, 2022. 2022.
Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
Ricky T. Q Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations, 2019.
Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuous-time transformer for irregular time series modeling, 2024.
Samuel N Cohen, Christoph Reisinger, and Sheng Wang. Arbitrage-free neural-sde market models. Applied Mathematical Finance, 30(1):1-46, 2023.
Christian Fabian, Kai Cui, and Heinz Koeppl. Learning sparse graphon mean field games. In
International Conference Artificial Intelligence and Statistics, pp. 4514-4514. PMLR, 2023.
xi

---

Published as a conference paper at ICLR 2026
Olivier D Faugeras, Jonathan D Touboul, and Bruno Cessac. A constructive mean- mean--ieel analysis of multi population neural networks with random synaptic weights and stochastic inputs. Frontiers in computational neuroscience, 3:323, 2009.
Shuang Gao and Peter Caines. Spectral representations of graphons very large network systems control. In 2019 IEEE 58th conference on decision and Control (CDC), pp. 5068-5075. IEEE, 2019.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2024.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In The International Conference on Learning Representations (ICLR), 2022.
Samuel Holt, Zhaozhi Qian, and Mihaela der Schaar. Neural laplace: Learning diverse classes of differential equations in the laplace domain, 2022.
Valerii lakovlev, Markus Heinonen, and Harri Lähdesmäki. Learning space-time continuous latent neural pdes from partially observed states, Advances in Neural Information Processing Systems, 36, 2024.
Sheo Yon Jhin, Seojin Kim, and Noseong Park. Addressing prediction delays in time series forecasting: A continuous gru approach with derivative regularization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, volume 33 of KDD '24, pp. 1234-1245.
ACM, August 2024. doi: 10. 10.1145/3637528.3677 1969.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons, Neural controlled differential equations for irregular time series, 2020.
Takeshi Koshizuka and Issei Sato. Neural lagrangian schrödinger bridge: Diffusion modeling for population dynamics, 2023.
Jean-Michel Lasry and Pierre-Louis Lions. Mean field games. Japanese journaal of mathematics, (1):229-260, 2007.
Mathieu Lauriere. Numerical methods for mean field games and mean field type control. Mean field games, 78(221-282), 2021.
Chenhao Li, Elijah Stanger-Jones, Steve Heim, and Sangbae Kim. Fid: Fourier latent dynamics for structured motion representation and learning, 2024.
Xuechen Li. Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. Scalable gradients for stochastic differential equations, 2020.
Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Chalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, Erez Shwartz, Mor Zusman, and Yoav Shoham. Jamba: A hybrid transformer-mamba language model, 2024. URL http:: httppi.////x.v..//////p/// wwg.wiwwww.o..oooooooooooooo.oot abs 403.19888
Guan-Horng Liu, Tianrong Chen, Oswin So, and Evangelos A. Theodorou. Deep generalized schrödinger bridge, 2022.
Ming Min and Ruimeng Hu. Signatured deep fictitious play for mean field games with common noise. In International Conference on Machine Learning, pp. 7736-7747. PMLR, 2021.
Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probability, 29(2):429-444 1997.
John W Negele. The mean-field theory of nuclear structure and dynamics. Reviews of Modern Physics, 54(4):913, 1982.
YongKyung Oh, Dongyoung Lim, and Sungil Kim. Stable neural stochastic differential equations in analyzing irregular time series data, 2024.
xii

---

Published as a conference paper at ICLR 2026
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2020.
Sungwoo Park, Byoungwoo Park, Moontae Lee, and Changhee Lee. Neural stochastic differential games for time-series analysis. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 27269 27293. PMLR, 23-29 Jul 2023.
Sungwoo Park, Dongjun Kim, and Ahmed Alaa. Mean-field chaos diffusion models, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023.
Christoph Reisinger and Wolfgang Stockinger, An adaptive culer-maruy maruyama scheme for mckean- vlasov sdes with super-linear growth and application the mean-field fitzhugh-nagumo model. Journal of Computational and Applied Mathematics, 400:1 13725, 2022.
Yulia Rubanova, Ricky T. Q. Chen, and David Duvenaud. Latent odes for ireeularly-sampled time series, 2019.
Mona Schirmer, Mazin Eltayeb, Stefan Lessmann, and Maja Rudolph. Modeling irregular time series with continuous recurrent units, 2022.
Ikaro Silva, George Moody, Daniel Scott, Leo A Celi, and Roger Mark. Predicting In-Hospital mortality of ICU patients: The PhysioNet/Compting in cardiology challenge 2012. Comput Cardiol (2010), 39:245-225 248, 2012.
Yang Song, Jascha Sohl-Dickstein Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben
Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
Belinda Tzen and Maxim Raginsky. Neural stochastic differential equations: Deep latent gaussian
models in the diffusion limit, 2019.
Ulrike von Luxburg and Olivier Bousquet. Distance-based classification with lipschitz functions. J. Mach. Learn. Res., ((Jun))669-699) 2004.
Jon Wellner et al. Weak convergence and empirical processes: with applications to statistics. Springer Science Business Media, 2013.
Jiongmin Yong and Xun Yu Zhou. Stochastic controls: Hamiltonian systems and HJB equations,
volume 43. Springer Science & Business Media, 2012.
Zheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang, Yifan Gong, Xuan Shen, Xue Lin, Pu Zhao, and Yanzhi Wang. Rethinking token reduction for state space models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, 2024. Association for Computational Linguistics.
Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on air-quality improvement beijing. Proc Math Phys Eng Sci, 473(2205): September 2017.
xiii

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 NOTATION TABLE
For convenience and to improve readability, summarrzes the main symbols used throughout the paper, together with brief descriptions of their roles in the mean-field formulation and learning algorithm.
Table Summary of frequently used notation.
Symbol
Meaning
T 00, T
Global time horizon sequence instance Past observation interval used as input Time label for past observations, w E Prediction time index, E T
O C T
"
I
p(u)
Distribution of observation times on O Joint distribution times and observed values Observed value at time u or t Data dimension of each observation Mean field predictor at (u, t) under control Law (t) at time for label u
p(u,y)
Yu:
d
(a())
(u(
va
Flow of laws (v (v,()) induced by control Mean field limit obtained by averaging over u at time Number of sampled predictors used in the finite system Empirical law of N predictors at time
Pu
N
(2(IR)
Probability measures with finite second moment on Rd Neural graphon weight between labels u and Feature interaction function between states and x Drift of the controlled SDE (t, Diffusion scale of the SDE at time f Brownian motion driving the stochastic dynamics Feedback control policy parameterized by 0 Trainable neural network parameters Set of admissible control policies
(w,,v))
b(t,
W,
(t,
0
A
Dr
Aggregated prediction obtained by averaging over labels u Cost functional of the mean field control problem
J(m
V(t,z)
Value function associated with the control problem
H(L,,, a, a.
Stochastic Hamiltonian in the Pontryagin principle 2- W-sssersteinn distance on (R)))
W2(.))
Generalized Wasserstein distance between flows of laws
Projector operator that updates the state law forward in time
Updater operator that refines the control via FBSDE
Fictitious play or gradient descent iteration index
xiv

---

Published as a conference paper at ICLR 2026
A.2 MATHEMATICAL BACKGROUNDS AND DEFINITIONS
This section includes brief summary of the mathematical backgrounds, omitted notations and defini- tions in the manuseript. Throughout, bold-face notation will be employed without loss of generality to omit subscript and superscript indices of mathematical entities where suitable for the sake of simplicity.
Generalized Wasserstein Distance. Recall the definition of space for probability measures that consist of generic path measures with finite second moments,
M = (v = (Va : a EO) E () uu E (), is measurable),
M v sup ueo
(X,,, < co).
For the arbitrary elements μ. E M: M M, let us consider M equipped with the generalized 2- Wasserstein metric as
sup sup Y_())) 1/2
(La((X.) P- n Law((Y.) P--
WLm(u, = sup uED
inf
E
(8)
where II is coupling between two probability measures and P. denotes canonical projection onto the interval O. Followed by the Battorovich-Rubinstere duality, definition in Eq (8) can be further modified as
J fd(pu,t
fd(puat- - Mase 11, EM.
2 sup sup uED reLip(L)
Note that the inner supremum is taken over family of L-Lipschitz real-valued continuous functions.
Cut Norm of Graphon. The cut-norm measures the discrepancy between two graphons over all possible cuts of the square of O. Formally, for graphon W 0 O + R+, the cut-norm is defined as:
W = sup / A.BCO
JAXB
Wu,, v)dudy
(10)
where the supremum is taken over all measurable subsets A and B. The definition illustrates that the cut-norm quantifies the maximum deviation of W from zero over any rectangle O2. Given the definition, one defines the metric called cut distance:
dg (W1, W2) = |[W - W
(11)
The cut distance measures how close two graphons are after optimally aligning their domains. the distance between two graphons W, and W2 is small, the graphs they represent structurally similar.
(Exponential AM-GM Inequality). For the arbitrary random variables X, Y and positive constants a, > 0, the expectation can be decomposed as follows:
Elexp(ax² + bY < (2E(cxp(2xX))) 1/2 (2E(cxp(2X3))) 1/2
(12)
(Arithmetic AM-GM Inequality). For arbitrary positive constants T, y, w > 0, we have
1
ry S WT. + 4w y-
(13)
XV

---

Published as a conference paper at ICLR 2026
A.3 ASSUMPTION
Without additional information, we make the following assumptions in this paper.
1. (H1). There exists finite collection of intervals {Ou; k
(1,..,, N3) for arbitrary E N+ such that U O2 = O. Then we assume the following:
For each k, the initial datum of the graphon system set with the data distribution Vu: O₂ uu (0) = Vu E P2, where the mapping assigns to independent measures.
2. (H2). For each & and Ok u. there exists constant C₁ such that we have probability Mu,e ( - Y|| -P P < Ci] almost surely for all p E N+, and the second moment
(2.)..m) of Vu,, is bounded.
3. (H8). The Lipschitz constants of the functions in modeling of graphons (() (t() 5 A + R+ are bounded above. The parameterized Markovian feedback controls are Lipschitz
in parameters:
( Lipw l| -llu.t)) {|| Bl,,, [ta 0a) - xt,, 0g)l1} - Lipe 2 ع,ll
(()
(14)
(15)
The drift function is Lipschitz continuous and dissipative, ensuring that the constant ci is well-defined.
(b bw (1,, x, cz) (b, (1, S Lipo - yllæ + ||a Bllma())-
(16)
C = inf -xx y) [(b, bx)(x) - (b, bw by)( /||x -yl
(17)
4. (H4). The maximal rank of embedding of neural agents A is d'.
T x θ a L2(V).
(18)
A.4 PROOFS
A.4.1
STOCHASTIC OPTIMAL CONTROL, MEAN-FIELD FBSDES
Before presenting the main proofs, this section offers a detailed analysis of how the proposed mean-field games can be formulated.
Weak Formulation of Mean-field Games. We start by explicating on the rigorous definition of forward mean- mean-field dynamic in Eq. (1) cost functional in Eq. (3) and gradient system of FBSDEs in Propsoition [A.2, followed by a brief summary of how forward-backward SDEs are formulated in the context of stochastic optimal control problems. To this end, let us first define the primitive process X,, which solves the following SDE for fixed label u::
dt.(tt) o,dB", Xo =y.
(19)
where By is Brownian motion under probability measure P. Given the square of volatility term of bounded below some constant, we introduce the probability measure pa.a, which can derived by the following Radon-Nine Radon-Nikodym derivative:
dip dP
-E
( (),
dB,
(20)
t=
where E denotes Doléans- Boléans-Dade exponential of martingale. Applying Girsanov's theorem, we have the Brownian motion Wr. under the probability measure Ipr.a.
W BB [0,- (b, b(s, (s). ds.
(21)
Then, the primitive process can be rewritten as follows almost surely puaaa
()),()) (X,,)), b(6 (x))) di o.dWinna
(22)
xvi

---

Published as a conference paper at ICLR 2026
By suppressing the objects in upper-seripts for simplicity, with the notation W Wi and (t) (t), one can recover the original mean-fiee -iel-fiedd forward SDE defined in Eq (.. Note that this formulation reveals that the process (t) is weak solution under [PA.a and the cost functional can be posed as follows:
J(v = { Bav Buxpp()) (t) ye vel
dt,
(23)
where the expectation Ea, is taken with respect pn.a, Note that the cost functional in Eq. 3 is an alternative form of Eq. (23). Next, we reformulate the approximation mean-field games with graphon in the probabilistic sense. Let att, x; 0) a(f,x, (. = â be extended control with fixed arguments Jr, e. For the fixed "i a.e., VUnif associated with the optimal control &', let us consider Hamiltonian-JacoiiBBee Bellman equation (HJBE), having classical value function V:
(t, Tr V(t, + H (1,x, O,V(t,,x) a (t, axV(t.x)) 0, (24)
Then, howarddbackward SDEs associated with the Hamiltonian system in 24 can be described in the Proposition A.1
Proposition A.1. (Weak Formulation: Forward Backwarrd SDEs 1) (Carmona & Delarue 2013
For the fixed flow of measures vu(-) :) : - P2 and the fixed label u, let (Xu(t), utt)). Zu(t)) be family of processes that solves forward-buckward stochastic differential equations with respect the proposed graphon system in Eq ① given follows:
(1) = (bw (X, (1), ②") + bl, X. â" dt o.dW", dt = - (t,, (t),,, Yu(t), Vu, & dt + (f( dWP
(25)
(26)
where bw (x, 0. = (W (w), (x, a.) the graphon interaction term, and terminal constraint
is given as Y (T) = G(XT, VT). Then, under the mild assumption (e.g., smooth boundness of 0, V
and Ozn V), there exist solutions of stochastic optimal control of the following minimization problem:
inf [v YY.0).. a
(27)
For the closed Markovian control such as neural control introduced in Section 2 the solution to adjoint process Zu (t) can be defined stated in Definition A.2, By rewriting
SDEs in Eq (25) and Eq (26) for non-optimal neural controls a (i.e., neural networks) which are updated via gradient descent, we can recover the proposed gradeint system of FBSDEs Definition 3.2
Definition A.2. (Gradient System of FBSDEs). For the fixed of measures v-(-)) : P2 and the fixed label u at each stage m, we consider a family of processes (Xu(t). Yu(t), Zu(t)) that solves forward-backen backward stochastic differential equations with respect to the proposed graphon system in Eg TT given as follows:
d. mmm (1) = bw (X (1), Vu, Om ddl + b(l,,,,, X am (1), am)d + o.dWa, u d a. (t) -H (t, X xmmaa (t), ym.a.. (t), Vu; 0m dd Z dW
(28)
(29)
O = ax (1, Xmm.a fm Rx.IST l." Vo a..a... (4)) E A
(30)
Law(X" m-
(31)
where 7th > 0 is a learning rate of gradient descent, and & is a set of admissible neural agents. Then, we have (Y,(t), (.(T),Z.(t)) (J, G, (a.J)o,
xvii

---

Published as a conference paper at ICLR 2026
A.4.2 ANALYSIS ON STOCHASTIC OPTIMALITY AND CONVERGENCE
Stochastic Optimality. In the following, we introduce the second type of forward-backward SDEs, which is based on the principles of stochastic maximum principle:
Proposition A.3. (Stochastic Maximum Principle: Forward-Backwe Backward SDEs II) (Bensoussan
et al. 2013) For the fixed flow of measures va(-) : T - P2 and the fixed label u, let (Xu(t), Y (t), Za (t)) be a family of processes that solves forward-backwan stochastic dif- ferential equations with respect to the proposed graphon system in Eq ① given as follows:
(X)(1) (bw (1w((x(()) (X,()), Mu: δ²) + b(t, X., (t), xx)))) dt + o.dW
dY (t) - H (t), Y (t), â a Z dW.
For the progressively measurable admissible Markovian neural control under the mild assumption (e.g., smooth boundness of 0, and Oza V), there exists a constant TSMP 0 such that the following inequality holds:
Ji a²) + TSMP /1Ia" - BBludtt S [[V
(32)
Remark. Note that the backward dynamics Y differs from the original backward dynamics Y, in Definition (A.2) as the dynamics is designed to be associated with Pontryagin stochastic maximum principle. This principle plays a central role in the proof of Proposition A.4 demonstrating the
stochastic optimality of neural agents in the following section.
In what it follows, we demonstrate that the stochastic optimality of the proposed gradient system can be guaranteed under the specific conditions required for constructing the control set in Prop A.4
Proposition A.4. (Maximum Principle of Graphon Mean-field System) Assume that there exists [H|ll,... KH Then, there exists convex set of admissible neural agents Oum E A such that the following relation holds:
constant K h such that ||0
DaJ(v am))
lim d de
d.
[amm + =((0m 0m-11) m m-+00 0.
(33)
c-0
Furthermore, the sequence of control profile {am} leads to the minimization of the stochastic Hamiltonian system in terms of Pontryagin maximum principle:
lim H (1), Y.M. (P,MM(()) Mus Ocm) 12-000
inf (1, (.)),, YMP (1), Mu; ox), di dl a.e. (34) OEEA
where the population is Vu ( T (1/a In other words, the value function can be derived by the proposed gradient system of FBSDEs:
inf J(V ai) = lim Ivvan am))
(35)
aeA
m-
Proof. We divide the proof into two separate steps.
1. Computation of Gâteaux derivative D.J. The aim of the first step is to provide an explicit computation of the Gâteaux derivative of cost functional (value function) with respect to the neural agent. To achieve this, introduce the variation equation and its associated gradient system of SDES with fixed B:
dY (f) - Y.M.MP (t), Vu, 2m dt + Z,m ddW"
(36)
di, = 10,bb + (L) + [10. bw + O. b)
(37)
(1) = d(i,(() Y.m. (1) Rd.
(38)
Let Y (m, e) = am + eBm represent the infinitesimal changes the admissible neural agent a" in the direction of B Cim Omm To feasibly select the convex combination T. (m,, e) for any
xviii

---

Published as a conference paper at ICLR 2026
and c E [0, 1), both neural agents need lie within some convex Am. For now, we assume that there exists convex set Am that includes a and [3. The explicit form of this set will be clarified in the subsequent step. Given the definition, compute the derivative follows:
DaJ(v am) Ju ( (m, e))
(39)
=E 1
(1)( am &
where denote x(t,x, a) [t]] woll. While aT) can be identified with (T(T), apply the product rule to the third dynamics dja in Eq 38) to have variational form to induce ju (T):
(t)(t) (t) di. [t)(t) d (t) + Tr(dYa (1) (t)
T
Y (L) (0.bw + 0,b)) (() (0abw 0a (()d
(40)
O.G. (0,bw + 0,b)3,m + 0.G (0abw + dab)i
Combining Eq 39 with Eq 440 and Cauchy- Schwarz inequality gives explicit form for the Gåteaux
derivative of objective functional.
Da (T c)) = J. 0a (1, [1, Y. (t), (T),,,(( (m amm )dt 3m
/ 100 H (t, X Y.m. ( ((
(41)
$
( am)) l. |1Bm le
< |1 Fz| || 1
where ||-IIp denotes Lormo and the last inequality is obtained by applying Hölder's inequality with conjugate pair (p = oo, = 1). Then, we have
1.1 = 1||laam - am - |llll (Il X 0t) ) att, X 0m--))(l()) ">
(42)
Y Y = V Y Y 1lp-aamm-- (1)]Ee.
Construction of A. Next, we define the explicit form of the control Am. The constructed control set must meet two conditions: (1) it must be convex, and (2) the right-hand side of the inequality in Eq (T)) must converge. For properly dealing with the first condition, let us consider metric ball B,, in L1 space as follows:
L1, I'm := Fu.tm = Lip, Y 1t).
Bm == B(am
(43)
E []
(44)
Since any arbitrary metric ball convex and the calculated reverse direction of gradient guarantees local minimum at each stage, the setup of the proposed metric ball ensures the well-definedness of Gâteaux derivative in Eq (41 and local optimality at each stage m.
Let (m) be an eigenvalue with respect to the principal direction of Hessian for cost func- tional, f.e., Hess (vo ()): Consider another control set C, := (am-1i 1; ( S (7 11... The conventional analysis of gradient descent gives the following inequality on C..
mmm
SY 1oym-1lam--
2 (2
110--11--11111))--- 11)2 (ammm) (Eóo Ym2
(45)
While the second term in right-hand side of Eq 45 non-negative, the sequence expec- tations for the backward dynamics is non-increasing, demonstrating that limm- DaJ < limm--ooo ES m = 0 when the infinite sequence { } lies within limm-xxe C... To inherit afore- mentioned properties lying in both control profiles for all m, define A. = (Bm Cm). where A lim to Am- The result directly follows from findings in the stochastic maximum
xix

---

Published as a conference paper at ICLR 2026
principle (SMP) (Carmona et al. 2018 following relation:
Bensoussan et al. 2013). ensuring the equivalence of the
Ed, H . 0 a' arg inf H .x).))
(46)
acA
Note that this equivalence relation is applicable only when A is constructed in the manner previously specified.
A.4.3 CONVERGENCE GRADIENT SYSTEM FBSDES, MEAN-FIELD EQUILIBRIUM
As we have formally defined the stochastic optimal control problem and established the corresponding optimality conditions, this section delves into the detailed rationale of how the proposed gradient descent-based FBSDEs achieve the Nash equilibrium. We will prove Proposition 3.5 through the following steps:
1. For the arbitrary probability measures (i.e., 12 va) associated with fixed Markovian con- trols and B, we first establish that the upper bounds of the generalized Wasserstein distance remain stable when two measure-valued operators and V are composed repeatedly:
( w em (uP)) [ o y ()))())))
mZM
(47)
2. Consequently, reparameterize reference measures (nº 2*) with the laws of inferred mean-field forward dynamics in Eq | at subsequent stages (i.e., vaat)) ), proving the convergence towards mean-field Nash equilibrium.
With the assumptions explored in the previous proof. for the fixed label u ~ p(u). the m-fold of composition & induces convergent behavior of generalized Wasserstein distance:
Proposition 3.5.
Wa( w/ 1),, [ o v cm 2)) sup W,, (1mm( 1 1/ 2)²
TET
C(T) (sup, supm Imm M -1 (OTT)M M-000 (su), supm I'm)
lim
sup Wr WM. waoe M M 0. (48)
M!
where numerical constant C is dependent on bu, C1, Hy. Lips, m2, |0,, e-11,-101, h, Lipw In other words, { y/oom is a Cauchy sequence on M, and the proposed gradient system converges.
Proof. Recall the definition of controlled graphon system that the particle dynamics at time / with distinctive controls a and B can be presented as follows:
X (t) ( + S." (Walved 35) (s))ds + b(s, b(s, (s), ad)ds + f a.dW".
- (0) //" (Weluv. 2) 8)) ds + S b(s, b(s, 3)ds f o.dW"
Given the dynamics above, the property of measure projection induces the upper bound of generalized Wasserstein distance as follows:
(L )()))) < E
sup |
( -x (s)]13
Lest
1 [11// ((X v)ddyss
bo
/ ((X" (s), Y W) v)du,. 2 dv
+ bo [ || X.ls. (s), a) b(s. x (t),
< 36p (I+ II III) + IV,
(49)
XX

---

Published as a conference paper at ICLR 2026
where the first and second inequalities are induced from Holder's inequality and the Burkholder- Doviiy (Chaintron & Diez 2022) with some constant by > 0. Following the assumptions in Section [A.3 and the modeling of graphons in Section 2 the first term (i.e., D) can be upper-bounded in the following estimation.
1= E Jo Ial (6( (xHH. (s), Y) Wo (u, v)dee,, mt) di
(50)
S'a / (u / (s) -s))((( (s)l duv.e ( (dvvvards)
< L
Given the fixed control a, optimizing the last inequality requires estimating the (local) Lipschitz continuity of positional encoding sb:
Lip(bb( ()) <
sup [/ Y)I xER
(51)
a-2
2(x
8 B (x 8)
Ia
a²
18
where ||x ||x- Y and @e denotes the Euclidean outer product. Following by the assumption
(H2). Grönwall's inequality with the fact that spec (Vb) == A S max(1, - -1aa= -2, we have
U sup x (r) x (r) ds
I I (3
(52)
Since each component 3b possesses the same spectral norm as s the second term can be upper- bounded with the improved definition of generalized Wasserstein distance in Eq (
II = 1/ 1oll/e (X (s). Y)Wa(u, v)dlur.- - (m..1(18/
dv
(EC d) 1 v
2
$ <d0 sup max
(x. (s), (ss)),))oo(l () - Mv,el
ds
(53)
C1
s <dDD/cbb(B) '' Wo.M(H² aa)dds
Regarding the third term (i.e., III), we have
III = R [ 1 (x),.((s), Y)I - Wa Idv (P)
/ dvuards
(2C7mally + 1/'² 1 WW Wal Walddv ddv Cnv (u, v)ds
/
(54)
I H2 + T (W Wa).
The upper-bound of last term can be directly obtained by the Lipschitz condition.
IV = E S b X..
(55)
< Lip,E 1// duu
X (r)]1 ds Lips l' suplla B11
r5e
xxi

---

Published as a conference paper at ICLR 2026
By replacing each term with numerical constants C3 C2. Cs in the aggregation of all four terms, finally have the following upper- upper-bounds bounds related to dg. WM and Ly-norm:
E
sup
X (5) (+)1
< 3b0 (I II+ III + (I()
I (3C H (3) + Lips) J sup rs*
x xx
Cs/tt)
+ C66b ma Hp + 36o)|T (W Wa)
(% supl|a 1 + (M. (a rSa
max(3bod(0/(1b((), Lip.)
(56)
Applying Grönwall's 's inequality to the above result in Eq (56) and the first inequality in Eq (49) shows that there exists constant C' = max(C3, C4, Cs) such that
W. ( ((1))) s c" (((WW):W)) + ( suplla Bll + W.m va) rse
(57)
Next, the aim is to show the upper-bound of dg. ||a aa- B1 and W.M. To proceed, let us first examine the upper bounds of the cut norms for both exponential and cosinusoidal graphons follows:
(W3 + 2Wo + W2,) + (2/L) WT. + W2,)
sup A,B
AxB (u, v)dudy S h((a)=
(T/2)) 10 -01-10 -1)
(58)
Modifying the upper-bound in Eq (58) by replacing Wa. with ow = Wa W3, one can derive the following
(W W.) - |8811 < max (ILLiPw, (T/2)(e-2TT)) 2 01 1)) 1aa - -1
(59)
At each stage {mJisssM with the given sequence of probability measures {vam) MI we substitute ((u²) and ((va) in Eq (57) with & (vaam1) and & o (p respectively. Then, one can derive the following relation:
W4 o am) 2o 2 (vam-1))) WLa coint (in1)) Lla(X"oa) ann))
C d( War )+/ 1' suplla (illlearr + Wo.n(vannt Dain) )2
rse
< C² max (t IlLipw,, ++ (T/2)(e-2)) (TT/)) -2T 110 1101 11)) (emlla(t;,mm-111 - alt, : 8T) 1|2
+ C'' [ (i. v
C max (t+ 11Lipw, (T/2)(E -2T 110) - -1)) sup
=C(1) S C(T)
C I( Wanne vm )
where the radius of metric ball (i.e., I'm = Fu,,m. was defined in the proof of Proposition A.4 In the first equality, the controls a are replaced with their optimal profiles a following the definiion of the operator Vi in. To set up the subsequent stage, we substitute a pair of controls (a ) with (0 Qm again. Next, we show the stability of the result obtained above for M- stage by
xxii

---

Published as a conference paper at ICLR 2026
observing the upper bound of M -fold of the operator composition.
WL. TOJM (w c T/jaM (2)o))²
S ((())sup sup I' + +C' f Wm. Wan.((D wiold (oold-1(a0) ( -/ooa-I (v
(60)
M Σ (C(t) s th + (C*)
I TM W. Wom.m( pam (ds ..... sM).
m==1
where d11 = ds @ ... & ds denotes m-product of Lebesgue measures (ds JssmgM. Finally, we deduce that the supremum of the left-hand side can be controlled by
lim sup We (( ())m( f ())m)))))) +00 tET
C(T) (sup, supm rm) -1
(C'T) M!
sup W +0 (61) IET
((T)p supm Im) -1
where the learning rate yth chosen such that sup, supm Im remains sufficiently small, and the last term in the inequality can be derived by modifying the following
(T (T-s) s) (m 1)!
sup We.M (4 (oom (Ma), ( (C')M
W. WoM )s
(62)
tET
The inequality Eq (67 demonstrates that the sequence operator compositions { v om JMSM M + M forms a Cauchy sequence, confirming the convergence of the proposed gradient system in the distributional sense.
A.4.4 SAMPLING ERRORS OF MEAN-FIELD PREDICTORS
Though not presented the manuscript, the following result implies key theoretical conclusions: demonstrates that the estimation enrors for the neural agent, introduced by the sampled mean-field predictors (empirical measure) at m-th gradient descent step, are kept within acceptable margins.
Proposition A.5. (Worst-case Estimation Error of Neural Agents) Let Qn Qn (u, (1/n) Σ, dxe, (t) and Q == vull) be empirical laws of mean-field predictors and their corresponding mean-field limit. Then, the worst-case estimation error can be upper bounded with probability at least 1 8:
/ (amd(Qn -Q)
2 5
3273(1 + m2)
sup an En
n
(d/2+2)-1
+ 4
32 2)/2 - (3d-22/22 11
(57 Lip.. V Y 1,am (1)llE)
mipp.
d/2 d+2 4(d 2)
(63)
Remark. While the admissible control set A guarantees the diminishing behavior of | m-,am (t) g, the second term in Eq (63) approaches zero becomes large, even when n is small.
Proof. The proof follows the standard convergence analysis of empirical processes. Let us fix the temporal variable t and the labels of mean-fiell predictors u. Then, one can show that the supremum of Euclidean norm can be decomposed follows:
Σ sup COEA
12,, F. am EENT aml S
Σ sup |Bq.. Eogl = F (Qm, Q), (64)
DEAL
xxiii

---

Published as a conference paper at ICLR 2026
where T 2 denotes the integral probability metric (Müller 1997) with respect to the set A which consists of j-th component of neural agents at m-th stage. Note that the the supremum in the second term is taken for all function 9 lying in the set of parameterized function, i.e, neural agent. Let us define P: (Rd)" - I such that (X (1),
X (L)) 2 sup, 1(1/n) o
Eggl, and (X (t), X (t)) I E, sup, (1/n) o.g(X (t)) where far Jign is a set of i.idd Rademacher random variables. Then both p and q satisfies the following inequality:
sup max (p, (4), ... X (1),x', iE(L.-m)
(1) X (4)
4T supx, xtt,x;
(p, 1).9)(Xx (t), X (t) S
(65)
n
Following by the McDiarmid's inequality with respect to p, we have two concentration inequalities:
-nE²
(p Ep 2 Pq
exp
2
(66)
87 supx, a xf,
Eq 2 2)
By applying the symmetrization inequality Wellner et 2013), we have the following inequality with probability at least -
8T supx, (el1, x: 0)2
1
T Adn (Qu,, Q
S
ET
QQ. Q) +
In
8
8T supx, att,, x; 0)2
1
S 2
sup yEA!
0gg (x +1
In
72
n
(67)
327 (1 + m2)
< 2 E.
sup
Σ o (x
In
Lo
(AL.. (X (0h)
where the outer expectation is taken with respect to the randomness mean-field predictors in the second line, and we apply McDiarmid's inequality in Eq (66) again to derive the last line. With the covering number of the Hilbert space for the L2-norm, we get
1 T1
Ry (As {xa, (t)))
& Eg
sup gEAL
32
00
< inf
2E +
V H(T, An (2(n))
(50
e/4
32 00
22m
d/2
< inf
20 +
dr
<>0
e//4
(68)
< inf <>0
2E +
32 (2rm)
d/2
4/2+1 (d/2 - 11-1 (E/4)
< inf 6>0
2e +
32 (3d-2)) 22/22 d/22
22/22 1//2-417 4/2-111 (d -222 2)-1
(d/2+2)-1
=4 <1(v
32 2 2(3d-2)12
(87 Lip,, 50 Yt Lip, - (1)) d/2 d + 2
4(d 2)
where we assume the data dimensionality is Theorem 16 (von Luxburg & Bousquet 2004). the second incquality can be derived from the fact that Qn an empirical measure, and A is a metric ball of radius r.m embedded on dinite-dimensional Hilbert space following by (H4). By setting d = d', the last result comes from the definition of radius Tm--
2. The second line direct consequence of
xxiv

---

Published as a conference paper at ICLR 2026
Proposition 4.2. (Sampling Complexity) Let vs ,ig probubility measures defined Eq 7 Then, there exist numerical constants c, c7, C8, Eg > 0, w 0 and 0 such that the probability squared Wasserstein distance can be controlled as follows:
P ( (T) 2 2 2
22
Ne E
Ne Ne-Ne (I (1- 128wb(a)
(1-
-d/8
1 724Ev/
(69)
N
Cg2 207 explcaetaiTt (cnI -1) ,o exp( exp(-Acs))
2 max
(70)
where u E 0, t is arbitrary and ((x)= = |Walls g is cut-norm of the proposed graphons (i.e.
exponential, cosinusoidal).
Eq 58
clarifies the explicit upper-bound bound of the cut-norm for the proposed graphons.
Remark. The approach used in the proof to establish the concentration bound is largely inspired by the series of works on the measure concentration (Bolley et al. 2007, [Bddhiraa & Fan 2017] Bayraktar Wu. 2022; BBarrktarr et al.] 2023 Bayraktar & Wul 2023), with slight modifications tailored to the structure of the proposed mean-held system. We intentionally omit some parts of the proofs in this work that have already been covered in the reference.
Proof. We divide the proof into separate steps.
Estimation of Concentration Inequality. For the controlled mean-field system via neural agents a, fix the the population v and its related control X = X,, and let = i/n n for the moment. First, let define the following probability measures:
n
== 1 n
n (x bu = m=f oi Σ Pu-s/m()). (71)
(x)()) (0): DI
Then, we analyze the law of difference between the following two mean-field dynamics:
(.(t) ((,t)=X,(0)) ( (Wall... (Walls.)))))Xx(()))) S. b(s. (s)) ( o.dWa,
(t) (0) + / (Waldvell (s) + ( b(s, a)ds a.dW((/m)
Given that fact that the expectation of Ito's differential for mean-square error can expressed as di A[t]2 ((A(t), d)dt + A(t)dW, + dtdt where ++ e and ma compensate and martingale part of A(t), we get
di (t)
(1) 28x(t)
b(s, (i/(),)))) dt
[
i 1
& (x.()) - E Wa i (2)v) & (x(Xx))(0)x))
W.
(72)
n j=1 n
where denote È. :
Ex and p(v) = ww [nif(()), 5X(() =x(/n((t) ((x()())x((())))))))))))))()))))
Then, the dissipativity assumption gives
dI((X((1)IB <I+
(73)
For simplicity let us denote Wis Wali/n, i/f), and Wiv (a v). Using the dissipativity of the proposed drift function. For the second first, one can get
(1 (b(s, (z), b(s, (4)()))) < - (
(74)
XXV

---

Published as a conference paper at ICLR 2026
By adding and subtracting new terms, we have
Whs [ (X), Xf) - (X(mm) XGmmm)]] (x(())
(75)
LiPs Σ
X
Similarly, the second term can be upper- bounded as follows:
wis 1b (X((X(())) ((m)) - E Bys (xx -oX(1)
(76)
(t) | 110 Fi
By adding and subtracting the term W (t), :), the fourth term can be improved as
who / (a(xx//(tt,)))))))))))) / Wii (X, (X:/o
vac (t) -x((1)
IV
Wis 1o (CW)avv),n()) + n2dg (Wis, Wir))
11
(6x(()) (|(
|ac Fiv n02, O.
(77)
Note that the last inequality tends to zero for large enough n. Aggregating all the terms and using the fact that g' g() & ag(() + b implies g(l) < f and d/di]]g(())) (1/2))())-+//g((,) where g(t) = (1/n) E. ( and (2Lip, c1), b = b(Fin- Fiv), we have
1 ni Σ
W D()
(t)l
(1Lip; (cc))(te))
sup "'3'
W
113
1
|
1 1P2
ds.
n
(78)
(ILip, -2c)
sup "'j'
11
Fi in
+
ds
n
VV+V
where the first inequality follows from the estimation of Wasserstein distance for empirical measures, and the last inequality can be derived by applying AM-GM inequality.
P [W3 A.) 2 c] SP
W
(2
e/2
+ P 2 e/4) +PVI+ 2 c/4).
(79)
VII
=0, n2
where the last term vanishes for small enough e, with large N.
xxvi

---

Published as a conference paper at ICLR 2026
2. Estimation of Exponential tial for the square norm of mean-field predictors. We first apply the Ito's lemma e for arbitrary scalar Aexp > 0 and observe that
In this step, derive the upper bound of the exponen-
dre Xa(Oili Aeape X.(Olli (2X, (b bw)dt + or + 27ssp (t)( o.dBBa)
(80)
where gradient and Laplace exponential can be calculated as Ve x (. 2 2xex (()ll
and A 1 2xeape (x (d 2 ( Taking expectation both sides with the dissipative condition, we can show that there exist constants (2 2dexp (-xc + oedexp).
Cg or that directly gives following inequalities
[ /M,dt
di djEle Acpl x. (0))
S E ( (0)) (c2 Xu((ll + cs) dt + E
(81)
+ No Notc, tc S (xx(s)) CIe
sup EST
(1)
supllvall t
(82)
where the second inequality is direct consequence of Grownall's inequality, and M, and N, denote some martingale. Applying Grownall's inequality again, we have the desired result.
Eo x.)) $ < (cs + Go (x)) l
(83)
B| x [1] < (exp(Nexpl yn [2) + c5) exp(cot) S (e7)²
(84)
where we used inequality et + e < exp (max(a, a, b) + In(1 + exp(-la - 6)) = (e7) such that Aesp [x, + cgT, In c5 cg... Note that the upper-bound of the term exp (Aepp (lylllyl1)))
at initial time t = 0 determines the exponential integrability of the right-hand side above.
Estimation of Probability P/V 2 c/4]. By the exponential Markov inequality with some constant > 0, Jensen's 's inequality, we obtain
PIV 2 e/4] = P
(iLip, 201)((-o)
sup W J' 113 P.
> |
ds ><//
e/4
xe/ E
c -1Lip,, -20))(-0))
<
(85)
Ab(aa)l Σ voa ((()) - Begra(t) V. ( (X)))) ))? ds
exp
Note that (x,,)l1) S (Ix + [lyl|=) have linear growth all x,y E Rdd by the assumptions.
E
exp
Ah(a:)
Σ 3 (X X(/n)) Eegia (X 11
<
E
exp
Lib N
X (1/a)) 113 +2b((a)
1
F2
(86)
n
n1
EE
4ih(
X X
1/2
12
1/2
<
2F
exp
2E
exp
2
Fab
where the last inequality can be derived by applying exponential AM-GM inequality
xxvii

---

Published as a conference paper at ICLR 2026
2
2v5
exp
26
F
=
exp
Fl ||
E
n
E
2
12
exp
w
Z
||
F
71
F
4w
(87)
2E
exp
[ n
| )1)
1/2
(2E
exp ((10n)
Lip (l 1/2 exp(cIef77
16w[
207
exp(c)e()T)
n
where Z ~ (N, a standard Gaussian random vector. The last inequality a direct consequence of the property of the moment generation function. The second line be deduced from the fact that the discretized predictors (s/n)) and ((/)) are iid with the selection w > 0, Xoxp and satisfying the following:
Fall In Lip X exp(cie(())
(88)
4w
(Ab(())Lipp (10n)Lip,)
Aexp max
(89)
n
Ç = 2xh(a)
(90)
By aggregating all the terms, we finally have
Ab(a)) _ Σ va (X(i))) X(/n)) Ee (X(mm)) 21(e
n
E
exp
(91)
S 2c (1- 16wL n12
exp(ciedeii)
Thus, the probability V larger than threshold c/4 can written as follows:
PPVV 2 e/4] <
2 E
ne 67 (1 16w(
exp(c4e2a1)) -1),
(92)
n²
- (4Lip, 201),
4n.
(93)
Estimation of Probability P[VII 2 e/2]. Now, it remains establish the upper bound of the probability related to VII. We modify the standard estimation of concentration probabilities of empirical measures as outlined in Bolley (201)). By the triangle inequality, the probability can be decomposed as
é
P
VII
sup
W2
D
6
+P
sup
PRA
>
(94)
2
hASIS(++1AA
6
0ShSM-1
6
0SNEM-1
where the temporal interval can be also decomposed as T [0.
- [hS, (h + 1)A]. The first term of the right-hand side above can be bounded as
P
sup
P
sup n osti
(x,((tz) Xtjn(te)
(h+11
(95)
< exp
-nsup <>0
E5 log Eexp
sup SS4
(t//(tt)) (t)nn
The first line is induced any measures vi) are empirical, and the next line can be induced by using Chebyshev's exponential inequality and the independence of the mean-field predictor. Denoting
xxviii

---

Published as a conference paper at ICLR 2026
(i/n) s /t (t((t))) (t1) Xs/n (t2)ll2 for any tyT, further improve the right hand side by showing
Exp (im) & exp((" exp (1/m)) & cxp( (1+ (1+
(96)
where we used the fact that ex I abb + 2ax for all a, b, 2 0. In order to show the upper bound of the first term in the last inequality (96), we used the result (4.6) Bolley (2010) tailored our case under the assumption made in Section A.3 for fixed and a. Combining results, we have
(cs- -nsup (5 - (2 - log((1 - CA1) <>0
P
sup
( PAA 2
< M exp
HASIS(H+1jA
0ShSM-1
(97)
S M exp
(-4cg
nE²
log(1 + +aA)
Cg
exp
ne² +
A = exp(Arr M S cg/e²
Acs
4cs
For the second term of the right-hand side in (94).
we first apply Boole's inequality of events to have
²²
2
P
sup 0ShgM-1
W2 PmD)
S
sup (SShSM-1
W2 Pma) 2
36
72
(98)
+ pP
sup
W hAs A
72
0ShSM-1
Me
Co
(72)* Vn S (72)
The second inequality can be deduced by the result of Theorem 1.5 Bolley (010) with d S d'
4, (0, 1) 8 2. p- = 2, g = Then, there exists a constant ni > such that n 2 nig max (e -16 16 .c) for any >> 0 and
2 <
sup IP (6x (t)
(99)
EET K
72
(72)*
where the quantity in (100)
be derived by proceeding similarly in Step 2.
sup
E (1(x) s oo
(100)
IET
The first term in the first inequality is direct consequence of following result:
E
(1/)(()) - X (1/m)())) (s) x |t 1 s/2.
(101)
Combining all the results for the probability bounds of V, VII for deduce the upper bound in 779.
P P (1) 2 c (72)*evn
Co
Cg + exp(- exp
ne 2
2
4cs
(1- 128wh(ax)
(1)288)((a))
e e 2222
3/2
exp(cre2a (EaT -1). (102)
By setting 2 as follows, the proof is complete.
2c 3/2
Cg, exp(caetaT) (CNT (caT-1) eg -4cs)
21 = max
(103)
xxix

---

Published as a conference paper at ICLR 2026
A.5 ALGORITHM
Algorithm Sampling Mean-field Continuous Sequence Predictors
while E do
P GGraphn Mean-field Buler-Maruyenr Sampling
while < N do (Uh, JiSN ~ p(u A, ~ p(A.), U ~ Unif(O), p(t). a, = ===(((,);0")) Wiy Wa, ([ru;] /n, [nvy] /n), b, = ba, (X)))
(104)
X + A.) (f) + E Wo A. b(t, X; a.) + N(O0. T.),Ta)
(105)
end while
Predict Subsequent Future Event
if E T O then
A4ta, = Σx w(U, [rus]] //)x (t + Ai) R Eusp(u) X Ac)
end if end while
Graphon Mean-field Euler-Maruyama Sampling. Algorithm 1 describes a discretization of the proposed infinite-order mean-field system. For a set of sampled temporal states, the proposed sampling method in Eq. (104) firstly projects the original graphon Wa and interaction functions ?/r onto their discrete counterparts Wiy and Vijy which are referred to as step graphons Fabiann et al. (2023) and step interactions in the literature. In the second phase, once the projections have been obtained, the Euler- Suler-Maruyama method is utilized to sample the trajectories of mean-field SDEs, effectively propagating information. the prediction interval i.e., T O, the aggregation function w is utilized to integrate the sampled particles, facilitating the generation of a forecast.
Continuity of Temporal States. is worth noting that every temporality integrated into the proposed framework is completely non-uniform, as minimal temporal granularity (i.e., A.), local and global temporal states (i.e., u, t) are distributed their corresponding probability densities defined on the continuous interval, resulting continuous representations of sequences. To operate in the described continuous setting, the neural network architecture our framework is temporally resolution-free freee which differs by exiting benchmarks, e.g., Contiformer. The following list summarizes
XXX

---

Published as a conference paper at ICLR 2026
One of forward evaluation
Shared Memory
Shared Memory
Shared Memory
Shared Memory
O00
O00
O0OO
by XX
by X
bu
Bu
Y
bi (
Yus
xx
iX.
Yo:
XX
122
Figure 6: Parallel Computation in Sampling Mean-field Predictors
A.6 EXPERIMENTAL DETAILS
Experimental Setup. We consider T = [0, 7) as the entire temporal interval for each sequence instance and use the first a% of observations, [0, (aT/100)), to predict the remaining (1 )),, [aT//00)) T". For this study, 7' is set to 100 for the MIT Humanoid Robot dataset, 48 for MIMIC-II, 72 for the Beijing Air Quality dataset, and 1200 for Eigenworm, with a 80 in all cases. For every dataset we construct such input output windows on the continuous trajectories and apply per-feature 2-score normalization using statistics computed on the training split, which are then reused for validation and test, and all models are evaluated primarily by mean squared error (MSE) on the prediction horizon. Our MFP trained with the Adam optimizer using batch size 128 and learning rate 10-1; the neural graphon and control networks together contain on the order of one million parameters, while competing methods use the architectures and hyperparameters recommended in their original papers rather than being artificially matched by parameter count. All models, including baselines, are trained for 0000 epochs without early stopping, and reported results are averaged over multiple runs with different random seeds single GPU. In mean mean-field game interpretation, the fictitious- siaitious-llyy index m coincides with the number of Adam gradient steps on the control parameters o, that these 10 000 epochs realize 10 000 fictitious-play updates and provide high-resolution approximation of the HJB component in Eq. (5) for this high-dimensional setting.
Model Architecture. In each forward step of (t()) (t) from to + At, neural network takes (t), t, and u as imputs and outputs a)) W(a). and w. In the first stage of the neural network, X. (t) and t are concatenated into single vector, which is then projected into hidden vector via multilayer perceptron (MLP). This hidden vector is subsequently passed through a computation block consisting of several MLP layers with skip connections. Finally, after the computation block, the hidden vector is projected into b(. o). W(x). and w using respective MLPs. In our architecture, each MLP is composed of two linear layers, with a Swish activation function positioned between them.
To process the labeling information in the neural network, we apply adaptive normalization (Peebles) Xie, (023). Specifically, instead of using fixed scale and shift parameters in the normalization layers of ((.,; (), regress these parameters based on The adaptive normalization layers are placed between MLP layers. We find that this conditioning mechanism effectively incorporates the labeling information, outperforming the approach of simply concatenating into input vectors.
After obtaining outputs from the neural networks, we evaluate bw a) for forward evaluation of SDEs. To derive bw(- a). we compute exponential or cosine graphon W using and where < (xxyy).. y) == (x-y)/||x y with x = X., (t) and y = (t). These values then integrated into with W(a) using Eq into W.. and a finally leading After Forward evaluation, we utilize w to aggregate predictors by applying softmax. (iee., A, E Ever Sof toff Softmaxww v); [w(t, u)
f. Next we calculate the projection Pxoj(x-3)
to bw(.,0 ax = Ever &. (X,(t)) (t))
where Softmax/ (x E S 5) represents the value of x after applying the softmax operation to the entire set S which includes x.)
Parallel Computation. Since the direct application of Alg. D is computationally intractable for large particle count N, we introduce novel parallel computing to efficiently sample proposed mean-field predictors, as described in Fig ?? At each step of forward evaluation, given all predictors X each predictor can be processed independently using Eq (L.. In other words, predictor needs wait for the others to complete their forward evaluation. By taking advantage of this property, at time t, we store all predictors with S i in the shared memory and forward predictors one step in parallel. This
xxxi

---

Published as a conference paper at ICLR 2026
parallel implementation significantly decreases empirical computation time by reducing the number of iterations for forward evaluation from (SSN) to (S)) where S is the number of steps for forward evaluation and N is the number of sampled observations.
xxxii


Reviews:
[{"review_id": "RnrlXq88ED", "reviewer": "Reviewer_c1GE", "summary": "This paper introduces Mean-Field Predictors (MFPs), a novel class of neural stochastic differential equation (SDE) models, for continuous-time sequence prediction. The core idea is to model the prediction problem as a mean-field game (MFG), where a continuum of predictors (agents) interact via a neural graphon to collectively predict future events. The authors develop a gradient-based training algorithm using forward-backward SDEs (FBSDEs) and provide both theoretical convergence guarantees and empirical validation.", "questions": "Could you provide a more intuitive explanation of why sequence prediction benefits from being framed as a mean-field game? How does this view improve over traditional ensembling or neural SDE approaches?\nHow about the impact of different ensemble-based predictors? Could we use simpler alternatives?\nHow about the computational cost of training interacting predictors?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Originality: The paper introduces a novel formulation of sequence prediction as a mean-field game, combining neural SDEs with neural graphons in a creative way.\nQuality: The methodology is technically sound with solid theoretical grounding and strong empirical results across multiple datasets.\nClarity: The paper is generally well-written, with clear definitions and helpful illustrations that support understanding.\nSignificance: The approach is broadly applicable to complex, irregular, and continuous-time sequence data, making it impactful for several domains.", "weaknesses": "The paper uses \\varepsilon-Nash equilibrium as a practical solution but lacks analysis on how \\varepsilon impacts prediction quality or how it behaves over time.\nKey recent models like continuous-time graph or attention-based architectures are missing from the comparisons.\nAll datasets are low-dimensional; results on high-dimensional or real-world multivariate sequences would strengthen the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pd9W1D3Nly", "reviewer": "Reviewer_t7CR", "summary": "This paper proposes a Mean-Field Neural Differential Equation (MFP) framework that reformulates continuous-time sequence forecasting as a mean-field game, and introduces a gradient-update algorithm based on forward–backward stochastic differential equations (FBSDEs) to compute Nash equilibria. On the theoretical side, it studies the coupled partial differential equations—Hamilton–Jacobi–Bellman (HJB) and Fokker–Planck–Kolmogorov (FPK)—and provides proofs of distributional convergence. Empirically, the method shows promising performance on the evaluated benchmarks, offering partial evidence of effectiveness and indicating potential research value.", "questions": "1. The introduction states the goal as modeling in the extreme regime Δt→0 and events→∞, yet the experiments and conclusions primarily show that MFPs perform well on standard discrete forecasting. Please clarify whether the actual objective is an extreme-regime continuous-time capability or competitive performance under discrete sampling.\n2. Apart from weaknesses 1, 2, and 6, the remaining listed weaknesses likewise raise concerns; please clarify each or provide supporting evidence.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. A nontrivial methodological contribution within the neural differential equation (NDE) literature: the paper frames continuous-time sequence modeling with a coherent design that explicitly targets irregular sampling and asynchronous events.\n2. Strong theoretical development with careful mathematical derivations, yielding a principled objective and training procedure; the technical depth increases the credibility of the modeling choices.\n3. On the evaluated (discrete, finitely sampled) benchmarks, the method attains competitive performance, indicating practical promise for continuous-time parameterizations (notwithstanding the limitations noted below).\n4. The problem is central to continuous-time ML/NDEs and is timely for the community, with potential downstream impact on temporal point processes, latent dynamics, and long-horizon forecasting.", "weaknesses": "1. Template noncompliance: multiple figures (e.g., Fig. 3) place captions above the panels, and Table 2 does not follow the ICLR table style; these violate camera-ready formatting requirements and must be fixed.\n2. Heavy notation with scattered definitions: the symbol usage lacks a consolidated reference, hurting readability. Provide an appendix table (symbol, meaning, shape/dimension, domain, scalar/vector) to standardize notation.\n3. Reproducibility gap: no code or artifacts are released. Public implementations (training scripts/configs, environment, seeds, and optionally checkpoints) are necessary to substantiate the claims and enable adoption.\n4. Evidence does not support the stated extreme-regime claim (Δt→0, events→∞): the experiments operate on discretely and finitely sampled datasets, without convergence studies vs. step size/solver tolerance, stress tests in dense-event regimes, or analyses of error accumulation in long-horizon rollouts. As written, results demonstrate performance in standard discrete settings rather than the claimed limit.\n5. Text–figure inconsistency: Ablation Study I states long-term prediction on EigenWorm, while Fig. 4 is titled “Ablation studies on the MIT Humanoid Robot dataset,” creating a contradiction that undermines confidence in the reporting.\n6. Missing experimental setup details and pointers to the appendix: data preprocessing/windowing/normalization, evaluation protocol and metrics, solver choice and tolerances, training hyperparameters (optimizer, schedule, batch size), random seeds, compute budget, and early-stopping criteria are not clearly specified, making it hard to assess fairness and reliability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m48nSrk5iW", "reviewer": "Reviewer_pvyM", "summary": "Unable to assess the work's impact/relevance, abstaining from reviewing process", "questions": "Unable to assess the work's impact/relevance, abstaining from reviewing process", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Unable to assess the work's impact/relevance, abstaining from reviewing process", "weaknesses": "Unable to assess the work's impact/relevance, abstaining from reviewing process", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)