Globally aware optimization with resurgence
Authors: Wei Bu
Venue: ICLR 2026 Poster (2026)
Keywords: non-convex optimization
optimization landscape
Primary area: optimization
TL;DR: We use resurgence--a perturbative technique to extract objective function value at critical points in high dimensional optimization landscape, which could serve as global targets for non-convex optimization problems.

Abstract: Modern optimization faces a fundamental challenge: local gradient-based methods provide no global information about the objective function $L$ landscape, often leading to suboptimal convergence and sensitivity to initialization. We introduce a novel optimization framework that leverages resurgence theory from complex analysis to extract global structural information from divergent asymptotic series. Our key insight is that the factorially divergent perturbative expansions of parameter space partition functions encode precise information about all critical objective function value in the landscape through their Borel transform singularities.

The algorithm works by computing the statistical mechanical partition function $Z(g) = \int e^{-L(\theta)/g} d\theta$ for small coupling $g\ll 1$, extracting its asymptotic series coefficients, and identifying Borel plane singularities that correspond one-to-one with critical objective function values. These target values provide global guidance to local optimizers, enabling principled learning rate adaptation and escape from suboptimal regions. Unlike heuristic adaptive methods, targets are theoretically grounded in the geometry of the optimization landscape.

Published as a conference paper at ICLR 2026
GLOBALLY AWARE OPTIMIZATION WITH RESURGENCE
Wei Bu
Department of Physics
Harvard University, Northeastern University Cambridge, MA 02138, USA wbu1123588 gma 1 11..i.... com
ABSTRACT
Modern optimization faces fundamental challenge: local gradient-based methods provide no global information about the objective function L landscape, often leading suboptimal convergence and sensitivity to initialization. We introduce a novel optimization framework that leverages resurgence theory from complex analysis extract global structural information from divergent asymptotic series. Our key insight is that the factorially divergent perturbative expansions of parameter space partition functions encode precise information about all critical objective function value in the landscape through their Borel transform singularities.
The algorithm works by computing the statistical mechanical partition function 7(g) = f ( for small coupling g < 1, extracting its asymptotic series coefficients, and identifying Borel plane singularities that correspond one-to-one with critical objective function values. These target values provide global guidance local optimizers, enabling principled learning rate adaptation and escape from suboptimal regions. Unlike heuristic adaptive methods, targets are theoretically grounded in the geometry of the optimization landscape.
INTRODUCTION
The main toolkit for non-convex optimization problems has been dominated by gradient-based optimization methods Jin all 2019); Fotopoulos al. (2024). The objective function landscape becomes increasingly non-convex and complicated making it an NP-hard problem to find global optimum, especially in modern machine learning where parameter space dimension is high Dauphin et al. 2014). Gradient-based optimization methods essentially offers step by step solution to the NP-hard problem, they inevitably suffer from a fundamental limitation: they are inherently local, providing no information about the global structure of the objective function landscape. This myopic view leads to well-known pathologies including sensitivity to initialization, convergence to suboptimal solutions, and the need for extensive hyperparameter tuning.
Current adaptive optimization methods like Adam Kingma & Ba (2017), AdamW, Muon Jordan ctt al. (2024); [hhen al. (2025) and other optimizers attempt to address these issues through heuristic momentum and learning rate schedules Zaheer et al. (2018); De et al. (2018); Chen et al. (2019 2022); Zaheer al. 2018). However, these approaches lack theoretical grounding the geometry of the optimization problem itself. They adapt to local curvature information but remain blind to the global structure that determines the locations and values of critical points- the fundamental objects that govern optimization dynamics. Another family of optimizers use the current objective function value adjust learning rates Polyak (1987): [oizou et al. (202T): Defazio & Mishchenko) (2023). given the optimum values of the objective function, which are usually inaccessible when the objective function is complex such as during neural network training.
The core difficulty in high dimensional non-convex optimization stems from the exponential complex- ity of the parameter space. For d parameters, exhaustive search requires O(2d) operations, making global optimization computationally intractable. This NP- uP-hardness forces practitioners to rely local methods that follow gradients without knowledge of where they lead or whether better solutions exist nearby.

---

Published as a conference paper at ICLR 2026
1.1 RESURGENCE THEORY: FROM DIVERGENT SERIES TO GLOBAL INFORMATION
Our key insight comes from resurgence theory, powerful mathematical framework developed by Jean Écalle for analyzing divergent asymptotic series Écalle (1881). In many areas of mathematical physics- ffom quantum field theory to statistical mechanies peee-prrrturttveeee expansions yield facto-
rially divergent series that nonetheless encode complete information about the underlying problem
Berry & Howls [1991); Mariño (2014); Dorigoni (2019); Delabaere & Pham (1999); Costin et al.
(2023) : Bhattacharya al. (2024).
The central observation that while these series cannot summed the usual sense, their divergence structure contains precise information about non-perturbative effects and critical configurations. Through the Borel transform and careful analysis of singularities the complex plane, one can extract exponentially small corrections that invisible to perturbative analysis but crucial for understanding global behavior.
Consider the seemingly pathological series En=o nlg". While this diverges for any g * its Borel transform Σx= 5 --x is perfectly well-behaved. The singularity at Ç = encodes information about the original function that generated the divergent series, and techniques from resurgence theory allow us to reconstruct the full solution including exponentially small corrections.
1.2 OUR CONTRIBUTION: SURGE
We introduce SURGE (Singularity Unified Resurgent Gradient Enhancement), using resurgence theory to locate the objective function values at critical points, which are further used as global guidance during optimization. Our method addresses the fundamental challenge of extracting global information from local computations by exploiting the mathematical structure of divergent asymptotic expansions. More elegant usage of these objective function value at critical points are yet to be dreamed up. We hope this manuscript serves invitation for further application of perturbative methods in the analysis of high dimension optimization landscape.
Key Innovation: We prove that the singularities of the Borel transform of the neural network partition function Z(g) ( corresponds exactly to critical objective function values in the optimization landscape. This provides a computable way to extract global targets from local information.
Algorithmic Framework: SURGE operates two phases:
1. Analysis Phase (performed once): Compute the partition function for small coupling param- eters, extract asymptotic series coefficients, and identify Borel singularities as optimization targets
2. Optimization Phase:
Use these targets to provide global guidance to any gradient-based optimizer through principled learning rate adaptation
Theoretical Guarantees: We establish rigorous correspondence between Borel singularities and
critical points, proving that our method captures global landscape structure. Unlike heuristic adaptive methods, SURGE's targets are mathematically grounded in the geometry of the objective function surface.
Practically, our experiments demonstrate consistent improvements of 15-30% in final objective function across diverse problems, from function approximation to large-scale neural networks. The method is optimizer-agnostic and requires minimal computational overhead beyond the one-time analysis phase.
MATHEMATICAL BACKGROUND
In this section, we give an introduction to resurgence with examples, hopefully this builds an intuition of how extract useful information from divergent series.
2

---

Published as a conference paper at ICLR 2026
2.1
AN INTUITIVE INTRO TO RESURGENCE
Consider the integral:
[(g)
dx
Ju
gz
We pretend not knowing the analytic form of this integral and for small g, attempt power series ansatz:
(g)
Σ ang
where On are coefficients that do not dependent on g. Computing the first few coefficients reveals
ag = a, = 1, 02 = 2, a3 = 6...., On = n!.
So our series is [(g) 1+ g 2y + +6y +24 + +..0=++..=>x.. nd! g" This series diverges for any 9 , 0! The coefficients grow like n!, completely defeating the polynomial suppression from powers of g 1, so the radius of convergence is zero. But readers familiar with special functions might recognize the original integral (g) to be perfectly well-defined for positive < 1. In fact, we compute it exactly:
((g) [ e d= 4/9 1/9)
1- gr
where r(00 is the incomplete gamma function. The obvious contradiction is that, how does divergent series represent convergent function In fact, such contradiction almost always arises when one attempts to use a perturbative expansion to probe the true solution by naively picking point of expansion in parameters space. the example integral we had, we have assumed that around == O, the integral behaves nicely. One can actually show that it is a saddle point of the integral, which is unstable under perturbation. The factorial divergence comes from the non-convexity of the parameter space landscape. We shall show this explicitly in the appendix
The series En n! is called an asymptotic series for [gg. This means:
N-1 Σ n! = 0 n=0
((g)
(4)
In other words, if truncate the series at the optimal point (before it starts diverging), the error is exponentially small. For our example, the optimal truncation occurs around z 1/g. At this point:
The terms are smallest: |aNg^1 2 |N!g N ee- me 1/9
The error is exponentially small: [[(g) Sx(g))
The missing exponentially small part contains crucial information about the function! That mean our truncated series almost as good the true functior
Divergent series are disasters mathematically. but we shall argue that they actually encode all the global information about the full function landscape in these higher order divergent terms. To make sense of these factorially divergent series, we introduce an ancient technique: resurgence
Given formal power series Ea ang" that asymptotic an n!, naive way make convergent is to divide it by n!! term by term, this is referred to as the Borel transform:
f(C)
(5)
'The famous Stirling approximation of factorial functions is another example where the coefficient grows 2(2)))
superexponentially 02y+1 (-1)² (2=)2r=2 which has zero radius of convergence. However, given finite n. we can compute its factorial (finite) exactly, hence the contradiction
*Intuitively, imagine we are perturbing around saddle point in high dimensions, in directions where our function is at the minima, it is safe to do this. However in directions where our function is at the maxima, the just about to fall off the cliff. 'The fashion we are introducing resurgence here is intuitive but not rigorous, however it is a strictly mathematically proved theory by Escalle Fscale 1981).
perturbation amplifies off cliff, the optimal truncation happens when

---

Published as a conference paper at ICLR 2026
For our example:
80 n! 5t
xc
f(C)
Σ
-
(6)
n!
1-5
==0
where have recognized the geometric series and the divergent series becomes convergent function!
To recover the original function, apply the Laplace transform:
T(g)= / c-Ilsi())ds [-= E
1-c dc
This integral gives us back the original function [g) due to the following identity:
= [( + 1) = Lo d'
(8)
In the original series, we simply take the n! part of On and write it in its integral representation:
20 ang
an n!!
g"
we effectively exchanged the integral and sum to obtain the Laplace transform of the sum of the geometric series. This is not a legitimate step since our original sum was divergent, but we see where it backfires at us in the example.
There is however one caveat, which we notice that the real line integration Ç is divergent since the integrand has pole on the real line (= 1. But what happens if we try to compute the Borel sum when the integration path hits this singularity? We simply need to complexify the integral and use integration contours that bypass the singularity.
Consider the integral:
Don
C/9
(g)
--C dc
(10)
When we slightly deform the integration contour above (+) or below (-) the real axis, we indeed by pass the singularity at ( < 1. However, this incurs an ambiguity of which contour to choose from as they give different results. Residue theorem allows us to compute their difference:
T </9
(o) (g) = 2 Resc=1
= 2mi 119
(11)
[+
Interestingly, this is also exponentially small, the discontinuity across the singularity is exponentially small (e- (e-1/s), but it's exactly the missing piece from the asymptotic expansion!
To resolve this ambiguity, we essentially add another term which also has the ambiguity but with an opposite sign. The complete solution involves trans-series a combination of power series and exponential terms:
x Ae
(gg)
n!g"
1/9
bmg m
(12)
perturbative
non-perturbative
where A 2mi is the residue the singularity called the Stokes constant and {bm} are coefficients expanded around the saddle point, in our case around Ç = 1. It can be obtained by expanding < = + /ou around u = 0. Here's the miraculous property: The coefficients {bm) in the non-perturbative part are completely determined by the original divergent series {an
The gist is, we have perturbative series which is Borel resummable across the entirety of the complex plane and give us the original integral back, but there is an ambiguity near the singularity, when the integration contour goes across the singularity, we see discontinuity in the result. This is
'The computations are done recursively using alien derivatives, we shall not elaborate on this point as this is the point of the paper.
4

---

Published as a conference paper at ICLR 2026
resolved by adding term that jumps in the opposite way (the non-perturbative term). Then we have a non-ambiguous definition of a series]
This means that terms in the divergent series. This is the essence of resurgence. We shall use these contour integration trick to localize onto the residue of the critical points of some objective function in section 2.3
extract the missing exponentially small correction from the higher order
2.2 BOREL-ECALLE RESURGENCE THEORY
Definition (Asymptotic Series). Let f(z) be a function defined in sector of the complex plane. An asymptotic series expansion of (z) as 0 is a formal power series
f(z)
Og 2
(13)
n=0
such that for any N 0,
N-1 f(2) Σ
an
= =0([z|
(14)
n=0
as - 0 within the sector.
Definition 2 (Borel Transform). Given an asymptotic series En=o On its Borel transform is defined as
B[J](C)
Σ
5"
(15)
T(n + 1)
where T is the gamma function.
The Borel transform converts divergent asymptotic series into convergent series (within its radius of convergence). The singularities of the Borel transform on the positive real axis correspond to non-perturbative effects and critical points in the original problem.
Theorem (Borel-Écalle Summation). If B()(() can be analytically continued to a function with singularities only on the positive real axis, then the original function can be recovered via the Laplace transform Écalle 1981):
f(2) 1 -BB((( di 1 E - -11/B[[1()/2dd
(16)
provided the integral converges.
2.3 STATISTICAL MECHANICS NEURAL NETWORKS
So far have only talked about this interesting trick recover useful convergence information from a perturbative power series that factorially divergent. the following discussions, we shall use in optimization, proving few theorems along the way.
Consider a neural network with parameters 0 E IRd and objective function T((0). We define the following quantity, which for physics audience simply the statistical mechanics partition function
(
L(0)/9 d0
(17)
Rd
where 9 0 is temperature-lite like coupling parameter
to moderate the behavior of the
partition function Geman & Geman (1984); Geyer & Thompson (1992); Kirkpatrick et al. (1983); Mandt et al. (2018). This trick was used widely in neural network optimization known Langevin dynamics/Gibbs sampling on non-convex optimization landscape Xu et al. (2020): Welling & Teh (201) 1): Raginsky al. (2017).
*This is done in the formal Borel-Fscall theory by computing the Lefschetz thimbles, we shall not delve into that.
5

---

Published as a conference paper at ICLR 2026
For small g, the partition function admits asymptotic expansion
(
Onl as g 0+
(18)
We show this in detail in 2d in appendix A As mentioned before, the coefficients an encode information about the geometry the objective function landscape, particularly near critical points.
Let us motivate this partition function definition with few explicit examples. A discrete objective function like L(0) the cross entropy objective function, we have true labels E {1,2,.... C} and network outputs po ())) Then the cross entropy objective function:
N
L(O) cross-entropy
log pa (Vi
(19)
So the partition function just gives
/ II (2)(())))) 1/9 do
[(g)
(20)
So a weighted partition distributed across different labels. When g >> 1, all contributions are suppressed, but decreases, the dominating modes show up saddle points in the saddle point approximation of the integral.
And for continuous objective function like the MSE objective function, we have the partition function:
7(g)= = exp
(co(x)) y1)² d0
(21)
2Ng
this can be seen as a distribution over Gaussian likelihood (f
N
Z(g)
/
II po po(vulfe(z;))dd
(22)
where this the log likelihood
p(ylfo(z)) = f(g
(23)
where the prior is uniform distribution over the parameter space 0. So (gg) can also be seen as Boltzmann distribution over KL divergence between the prior and posterior.
2.4 CONNECTION TO CRITICAL POINTS
In optimization problems, the central objects of interest are the critical points of the objective function. In this subsection, we shall state and prove some results that one-to-one relate critical points of the objective function to the singularities we discussed in the previous section.
Proposition 2 (Critical Point Correspondence). The singularities Ça of the Borel transform (CZ) on the positive real axis correspond to critical objective function values in the neural network landscape. Specifically, if 0 is a critical point with ((0*) = O, then L(0*) appears as a singularity of B (C) Bhattacharya al. (2024).
Theorem 3. The critical point contributions to the integral representation of an asymptotic series are in one-to-one correspondence with the poles on Borel plane or
do(xx
BlZ(g)](t)
J VL(x)]
(24)
where the integral is done on a level set in IR" L(x) with appropriate measure (x Bhat Bacharya et al. (2024).

---

Published as a conference paper at ICLR 2026
The derivation of this statement uses geometric theory which shall defer appendix C to not interrupt the flow. essentially transformed the computation of critical points of the objective function equivalently to searching for singularities function B (t) What is even better is the singularities 4, are exactly the value of the objective function at its critical point because of the level set constraint & [xx)..
the complex plane of the Borel transformed
This is a fact that can be extremely useful for optimization, since all existing popularized optimization techniques essentially local search using gradient or higher order gradients. This is tackle the NP hardness of searching through exponentially large parameter space. The downside of this that the updates are completely local and blind, demanding the optimizer to guess the learning rate stochastically (SGD Robbins & Monro (1951))o with momentum update (Adam[Kingma & Ba (2017). This Borel equivalence theorem instead offers glohal information(oojective function value at its critical points) about the optimization landscape. We investigate this further in the next section by developing a simple algorithm giving global guidance to local optimizers.
3 GLOBAL GUIDANCE AND TEST ON VARIOUS OPTIMIZATION PROBLEMS
Given the theoretical guarantee and discussions in the previous section, we pose the following algorithm for computing the optimization targets.
At initialization, we first compute the objective function numerically L R== Objective(fe, (Ix), yk) for set of initialized model parameters {0:) and data pairs ((
Compute the partition function Z(g) ~ + Em= e- by maximizing a concave lower bound function, trick we shall mention in the numerical implementation part equation 26
VN
Fit Zgg) to power series in g: Z(g) ag + aig + agg² .... aوgo up to order J.
Given {a the coefficients of the power series (non-vanishing and factorially divergent), find Borel singularity {Sm JMM in the function Σ== (a;/j!) the positive real line.
where the final step is searching for the target objective function values at critical points we can use as guidance during optimization. Since the objective function value real and positive, and one can already compute such value at initialization T.- we simply need to search through the interval (0, Lo) on the real axis, which usually not large. Before delving into the details further, we first note the following benefits of this algorithm:
Borel plane search space is always O(1) on the positive real line.
Only need to perform the analysis once at initialization.
Search for meaningful coupling range can be parallelized.
More specifically, discuss the algorithm mentioned above further. The detailed algorithmic representation is attached in appendix B The dominant complexity for large network is O(N2 where N is the order of derivatives computed, p is the network parameter and B is the batch size. So it is linear in network size, parallelization and other tricks are under exploration for scaling.
Numerically, we find that one needs to iterate this search for multiple different coupling parameters g in order to find numerically meaningful targets. One is required to determine the appropriate coupling parameter range where the partition function can be reliably computed and the asymptotic series extracted, algorithmically we implement algorithm 2 For given coupling parameter g. to compute the partition function numerically, naive way that works well for low dimensional parameter space simply uses Monte Carlo sampling Zgg) so (2mg) 4/2
Partition function computation
Σm e T/04)/9 where [ are samples drawn from appropriate distributions. This can be really inefficient in high dimensions thanks curse of dimensionality. Instead, adopt a trick
7

---

Published as a conference paper at ICLR 2026
using a sampler (flg) which can be whichever is the most convenient. Then simple rewrite
( = fexp exp
L(0)
+ (0g()v((1)))
dqu (0]g)
(25)
9
(v
We use the following inequality trick:
-log f Ev
(019) / -00
-c-
(ox0
+1
(26)
where optimality of the right hand side achieved when c = log f ED (0.9) dq This suits purpose perfectly. We simply need train separate neural network maximizing the following objective:
c,, - Egnge (.1s) lexp( L. c)
(27)
at maximum, we have
(²(g) log (gg)
(28)
This gives a robust way of estimating the partition function.
Given partition function values {Z(ga)}s= at coupling points {go 18 we extract the asymptotic series coefficients {az)ımo by solving the weighted least-squares problem:
min [as]
ws
7(g.)
(29)
where the weights chosen Wa 1/ (g. + E) to emphasize the small coupling regime.
Borel transform and singularity detection:
The Borel transform coefficients are computed as:
On
b
(30)
r'n +
We detect singularities using two complementary methods: First the ratio test, for convergent series,
the radius of convergence is given by R lim bni The dominant singularity is located
Ç R the positive real axis. We also direct evaluation test points {Cx) and identify N-1
singularities where
Σ D b > for some threshold T.
3.1 TARGET SELECTION AND OPTIMIZATION UPDATE
Definition 3 (Critical objective function Targets). The set of critical objective targets is defined as: T= (S E S(B(Z)) < S Lo}
(31)
where S(B)Z) denotes the set of singularities of the Borel transform.
Given set of objective function values critical points {S.. >= the ideal thing one could do is of course to reverse the objective function T(,)) G. and find the set of parameters 0 such that the objective function attains this particular value, however, this is precisely what the NP-hard problem involves. Instead, we still employ the usual gradient descent algorithm, updating the parameters according to the gradient the objective function, although we have in mind what value the critical objective function takes. During each step of gradient descent, simply check the list of targets for values lower than the current objective function Legurent and increase or decrease the learning rate accordingly. Mathematically, at optimization step I, select
Ge = max {( E T : Tecurrent)
(32)
Then simply change the learning rate with additional factor:
((++1) = (()) 17 a V ((((t)))
(33)
8

---

critical value Published as a conference paper at ICLR 2026
where
a (K) = 1 min (1 [(000) Ce T((10)
(11)
(34)
with some weight À > We see that when Çe target much lower than the current objective value, for example when we are stuck in a local minima, the second factor is close to A, hence the learning rate 1j scaled up by + À for bigger optimization step. Then when we are close L(O()) ~ St. we just use the local optimizer step without scaling up the learning rate, i.c. falling back to the original optimizer. In practice, the targets are chosen dynamically, for example, when the current loss value is close to the target within a certain threshold, we simply switch to the next biggest target in the list of targets. Additional ablation test in figure was also include in appendix E to test if randomized targets also accelerates the optimization process, this shows whether the targets computed using SURGE are actually meaningful and useful for guiding optimization.
We note that this can be generically applied to any optimizer as a wrapper function, where we simply multiply scaling factor in front of the learning rate in the optimize We further test this against existing learning rate schedulers on the Adam and present the results as a table and selected loss curves in figure i appendix
This can be viewed as a principled extension of adaptive moment methods with searetically-grounnde target selection. When Borel analysis fails, the algorithm gracefully degrades to the usual adaptive local search. Here we present the previous discussion as an algorithm:
Algorithm SURGE: Complete Algorithm
Require: Initial parameters 0o. objective function learning rate 1,, resurgence weight
1: Analysis Phase:
2: T BoreeAAnllyisi(Bu L), +
3: Optimization Phase:
4: while not converged do
5: f t + L inget + SelectTarget L(((t-1)),
6:
(1) + CompuccCuidance(((((
Langet A)
7: AO + - (t) Any optimizer update, A0 x- Clip(A8, max norm)
8: 0(9) + 0(t + A0
9: end while
10: return ((t)
4 EXPERIMENTAL VALIDATION
In this section, we experimentally verify the effect of having global guidance provided by SURGE during neural network optimizations. First simple function fitting then with MLP on MNIST and small transformer on the Shakespeare dataset across different learning rates. We shall see that compared the original optimizers, the SURGE wrapped ones accelerate initial convergence and offer quick escape of local minima.
4.1 FUNCTION APPROXIMATION BENCHMARKS
We evaluate the algorithm on simple fully connected network sized (12, 10, 8) fitting some ld function:
((x) sin(2x) + 0.5 cos(5x) + 0.3 si000)) + 0.
(35)
The results are attached as figure 2 in appendix
TThis is rather crude usage of the global targets, a first proof of concepts. We expect there be much profound ways of using these global targets.

---

Published as a conference paper at ICLR 2026
4.2 REAL DATASETS
We test on high dimensional neural networks optimizations with fully connected MLP parameters standard MNIST dataset in figure 3 and Shakespeare text training with a small transformer architecture ~ 10k parameters in figure 4 The SURGE wrapper is created with standard optimizers SGD, Adam, AdamW, Muon, the train/test loss function values vs training epoch are plotted in the diagrams in the appendix E For immediate visualization purpose, we demonstrate the comparison between Adam optimizer and its SURGE guided version on Shakespeare dataset training with small transformer. In figure 1 dotted lines are the SURGE wrapped optimizers compared to the bare
Adam All Learning Rotes
Adam Learning Rates
24810
4x312
Mare TUC2
13
26x
260407
14
14x
12"10"
20
106
Spack
(a) Adam with MNIST
(b) Adam with transformer
Figure 1: Comparison between Adam and its guided version
optimizers themselves. We found the following features of the algorithm:
SURGE accelerates initial convergence and fast escape of local minima
SURGE creates instability during the process, due to violent scaling of the learning rate
If the original optimization process generalizes poorly, SURGE will accelerate the overfitting, we present the observation in figure in appendix
the theory of resurgence obtain critical value of the objective function in an generic optimization problem. As a proof of concept, we implemented the set of critical values as global guidance about the objective function landscape. This is done in rather straightforward way by adjusting the learning rate of gradient iepss-implemented as wrapper on any optimizer-the system becomes "globally aware", making less susceptible to local minima and converges much faster than the base optimizer. We hope this manuscript serves as an invitation for further application of perturbative methods the analysis of high dimension optimization landscape.
Remarks: We have demonstrated how
an algorithm 3 that
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Michael Victor Berry and C. Howls. Hyperasymptoties for integrals with saddles. Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences, 454((892))657-6675 1991. doi: 10. 10.109%/spa..1911.0119 URL httpps///oppallccceyyaoo..oo....
doi Voi/abs/1es 10w.:...../pppp....m 11t...999111111
Arindam Bhattacharya, Jordan Cotler, Aurélien Dersy, and Matthew D. Schwartz. Renormalons as Saddle Points. 10 2024.
Congliang Chen, Li Shen, Fangyu Zou, and Wei Liu. Towards practical adam: Non-convexity, convergence theory, and mini-batch acceleration. Journal of Machine Learning Research, 23(229): 1-47, 2022. URL
org wttp://rrrrggpppe////eeaa
1432000-13--111 438. html
Xiangyi Chen, Sijia Liu, Ruoyu Sun, and Mingyi Hong. On the convergence of a class adam- type algorithms for non-convex optimization, 2019. /RL//ttts:///rrvvv.rr/////
02941
Ovidiu Costin, Gerald V. Dunne, Angus Gruen, and Sergei Gukov. Going to the other side via the resurgent bridge, 2023. URL httpt:ttpp//aatt wttp://wwwogg/b//////0000 12317
Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in nigh-dimensiona. non-convex optimization, 2014.
www.aawwwwwww..a./aa//aa.aaaaa// abcgg//bs//in 140...//11....0 2572
Soham De, Anirbit Mukherjee, and Enayat Ullah. Convergence guarantees for rmsprop and adam in non-convex optimization and an empirical comparison to nesterov acceleration, 2018. URL
https: wwtp:oog.//ooooooooooo.oooa.a
18011810000 06766
Aaron Defazio and Konstantin Mishchenko Learning-ran coaming-rate-free free learning by d-adaptation, 2023. URL
Eric Delabaere and Frédéric Pham. Resurgent methods in semi-classical asymptotics. Annales de P.I. Physique théorique, 11(1):1-94, 1999. URL hww.s:ttpppo/ //www..//wwwwww// nundam. org/ e
ATHPA 1999
Daniele Dorigoni. An introduction to resurgence, reans-seress and alien calculus. Annals of Physics, 409:167914 2019. doi: 10.101 1i..016j.oop2201..679999 Accessible survey for physicists covering resurgent analysis and alien calculus.
Greg B Fotopoulos, Paul Popovich, and Nicholas Hall Papadopoulos. Review non-convex optimiza- tion method for machine learning, 2024.
02017
Stuart Geman and Donald Geman. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Trans. Pattern Anal, Machine Intell., PAM1-666)7221-741, 1984. doi: 10.1109//PAMI.19994
1984.477799.
Charles J. Geyer and Elizabeth A. Thompson. Constrained monte carlo maximum likelihood for dependent data. Journal of the Royal Statistical Society. Series B (Methodological), 54(3):557-69, 1992. ISSN 00359246. UtLhttp:://
www. /
Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M. Kakade, and Michael I. Jordan. On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points, 2019. URL
attts:://axxi>: org abs 1902 04811
Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bemstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https
//kellerrordan.giieeeee cacttuuu.io/ppppe ic.posts/muon!
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. URL
https:
ww.ooorg//... abss 1412. 6980
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing, Science, 220 (4598):671-680) 1983. doi: 10. 2tt./sciinne.22.455532. 671. URL https: :ecttps:/wwwer //www. science arg/doi/abs///e 1 ence. 220.eee........... 40.00.4598.... 671

---

Published as a conference paper at ICLR 2026
Nicolas Loizou, Sharan Vaswani, Issam Laradji, and Simon Lacoste-Julien. Stochastic polyak step-size for sgd: An adaptive learning rate for fast convergence, 2021. URL https::/aaxxi:
2002. 10542
Stephan Mandt, Matthew D. Hoffman, and David M. Blei. Stochastic gradient descent approximate bayesian inference, 2018. URL httph: / org/ abs/ 04289
M. Mariño. Lectures on non-perturbative effects in large gauge theories, matrix models and strings. Fortschritte der Physik, 201400005. URLhttp://dde wwtpdxxxoo.ccooo org/ 10. 1002/prop 201400005
540, April 2014. ISSN 1521-3978. doi: 10. 10.1002/prop.
Boris T. Polyak. Introduction optimization. New York, Optimization Software, 1987.
Maxim Raginsky, Alexander Rakhlin, and Matus Telgarsky. Non-convex learning via stochastic gradient langevin dynamics; a nonasympiotic analysis, 2017. URL
/.bs/100..0 03849
Herbert Robbins and Sutton Monro. A Stochastic Approximation Method. The Annals of Math- ematical Statistics, 22(3):400 407, 1951. doi: 10. 10.1214/amm/1117779966 URL https
//doi.org 0. /214/aams///// /
Wei Shen, Ruichuan Huang. Minhui Huang, Cong Shen, and Jiawei Zhang. On the convergence abt.bbb5./5553
analysis of muon, 2025. URL https::/aaxxi..org///// /ttp:///aaa....//////5555.53333
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dy- namics. In International Conference on Machine Learning, 2011. URL https://api
semanticscholas wwt. htt.aaa.....reooooooo00:22211190
Pan Xu, Jinghui Chen, Difan Zou, and Quanquan Gu. Global convergence langevin dynamics based algorithms for nonconvex optimization, 2020. URL
06618
Manzil Zaheer, Sashank Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. Adaptive methods for nonconvex optimization. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-
Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https:: wttp::/inngg.errrr.......... cc/pper
Jean Écalle. Les fonctions résurgentes, Vols. I-I1I. Publications Mathématiques d' O'Orsy,, 1981.
Foundational work introducing resurgence theory and alien calculus.
12

---

Published as a conference paper at ICLR 2026
A LNEVITABLE DIVERGENCE OF PERTURBATIVE POWER SERIES
The coefficients (g) encode information about the geometry of the objective function landscape, particularly near critical points. This series is naturally asymptotic as we shall show it. According to the analysis done in Berry & Howls (991).
For simplicity of presentation, we demonstrate this 2d, where & IR, the higher dimension cases naturally follows. We shall pick the original point in objective function space we are expanding around L,, Then we first factor out the contribution of L,, and write the integral as an expansion around T,;:
Zgg) exp((Inn/9)
c (((1)) d0
(36)
(in(
We shall show that (g) diverges factorially. First we perform a change of variable u(f) = (L(0)-L,) for u + is actually a double valued function with 0(u)+ and ((u) Rewriting
the integral:
(g) = J. duc- (vro(w))) L'(O(u)-))
(37)
We recognize this as performing Laplace transform of some contour integral since:
1 f. d0 ((((0) - Ln //g)
(38)
T)((()))) T((0(u)-) 2/ Sr. T(O0) I.n gu
which is the residue the point of expansion Ln. We can expand this around g 0, which allows us to isolate the Laplace transform, which produces a Gamma function.
2 Σ
Z g'
I'r 2 2τi
J. do (L(O) [(((()) L,)
1
(39)
where the factorial divergence naturally appearing. In higher dimensions, the derivation follows from usual Stokes theorem. The divergence any perturbative series (approximation) suggests the existence of the existence of saddle local maxima critical points nearby, where divergence occurs when the integration contour goes through nearby critical points. Such effects suggest that by examining the divergent higher order terms in the expansion, extract information about these additional critical points. This is the essence of resurgence, here we use probe the objective function landscape in search for values at those critical points.
should be able to
B ALGORITHMS
Here we include the detailed algorithms described in the main text. The dynamical range search algorithm is as follows:
The main Borel singularity based target searching algorithm described in section
C PROOF FOR THEOREM 3
simple proof for this involves a trick in measure theory named the co-area formula, which used to reduce the dimension of an integral onto some lower dimensional domain level set.
J [(z)) S ( (()))
g( o(0)dH)-)(()) dt
(40)
where & n, u(x) labels the dimensional level set, [Jau(z)] det (Ju())J((z))) its k-dim Jacobian and d' represents the appropriate Hausdorff measure on the - dimensional level set. This comes from the simple fact that we can write measures:
dr J(, dt dHH.-(()))
(41)
13

---

Published as a conference paper at ICLR 2026
Algorithm 2 Dynamic Coupling Range Search
Require: Model parameters Oo. objective function L
Ensure: Optimal coupling range (Jmin; Imax) or failure
1: Litt < L(0g)
2: R + ((Leet 10", Let 10 : {-6, .5..... 1}}
3: best score 0, best range <- null
4: for (Ammin Imax) E R do
5: success praeeratee---- QuickTessttraannn Fmax)
6: if success 2 0.7 then
7: result < FullEvaluate (fmin Imax) if result.score > best_score then best_score result.score
8:
9:
10:
best range + (Amin Imax)
end if
12: end if
13:
end for
14:
return range
where fx) t defines the level set.
Using the co-area formula in the case of = 1. u(x) R" - R becomes a scalar function. And we denote the measure the dimensional level set using (z).).
I DERR g(x) Vu(x)d = ( ((z)dH(()) (1) dt
(42)
Multiplying both sides with delta function to indicate the level set 6(t - u(x)). Then we have
[ g(x))(( u(=)))
g(z) dH
(43)
Now choosing g(x) [ul[)] gives
| 8(1 0(1-
di((x)
(44)
Vu(x)]
We shall this formula in the following proof.
Writing the multi-variant function as an asymptotic series f(g) = ang". We denotes its Borel
transform and the corresponding inverse transform:
B(f(()
Σ
n
(45)
n!
n=U
And
B(B[f))()) = /- B((t) 1-
B/t
(46)
Jo
where a rescaling of the variables has been performed. an integral representation also exists:
f(g) = f -((x//
(47)
And on the perturbative level, the series f(g) = B(B(f))g) although with different radius of convergence.
Then we can write it on a level set then integrate over the choice of level:
1 f de 1/1 ( 0(1 S(x)))
f(g) =
(48)
14

---

Published as a conference paper at ICLR 2026
Algorithm Borel Singularity-Ban -asted Optimization Target Computation
Require: Model parameters {0: Y data pairs {(T*: M&) polynomial order J, number of
singularities M
Ensure: Optimization target values at critical points {Sm
1: // Step I: Initialize and compute objective
2: Compute initial objective: Lo = Σk=1 Objective (Tx); Mk)
3: // Step 2: Determine coupling range
4: Lief + Lo
5: Find optimal coupling range (Amins Imax) using Algorithm 2
6: // Step 3: Compute partition function
7: Select coupling points: {9x [9x)5=1 C [9min= 9max]
8: for = 1 to S do
9: Initialize sampler go (x (.)) and parameter
10: repeat
11: Sample {0, yemr (- (1gg)
Compute (f) 9x) = L(8,) + logg(g
12:
9,
13:
Update v, by maximizing:
14:
J C. = Esngs (O, go) - -c]]
15:
until convergence
16:
7(gs) < exp(ce)
17: end for
18: // Step 4: Fit power series
Solve weighted least-squares problem:
20: {azl/=0 + arg min, E w. ((ga) - -y
21: where w. = 1/ +e)
22: // Step 5: Compute Borel transform
23: for = to do
24: bn F(++1)
end for
26: // Step 6: Detect Borel singularities S
Initialize singularity
28: // Method 1: Ratio test
29: Compute R. 4- lim bn
30: if R < La and R > 0 then
31: S
32: end if
33: // Method 2: Direct evaluation
34: Define test points: {Sx (Gakkm Kuse
C (0, Lo)
for k 1 Kiett do
Compute B(Gx)= = Ei bask
36:
37: if B(Gx) > T and is local maximum then
38:
S (Cx)
39: end if
end for
41: // Step 7: Select optimization targets
Sort S by magnitude
43: Select top M singularities: { C S
44: return {mm 1M as optimization target values
15

---

Published as a conference paper at ICLR 2026
where we have already replaced the exponentiated S(x) with t. Using the co-area trick equation 44 we have
(0) (xr) [VS(2)]
f(0) - - -
1 % dt d/
(49)
where (x) is the appropriate measure. Comparing this with the Laplace transform formula equation 46 We see that the Borel transform can be written as
do (
B (f1
(50)
S-
VS(c)
It is easy to see that singularities of the Borel transform occur precisely when (2) = 0, which are critical points of the integral representation.
D A CONCRETE ANALYTIC EXAMPLE
D.1 PROBLEM SETUP AND EXACT SOLUTION
Consider the quartic oscillator partition function:
Z(g) =
dx e V(a)la, V(z)= - x²
(51)
This integral can be evaluated exactly in terms of special functions. Using the substitution u = and properties of the gamma function:
x²
(gg) 2 //000 du
(u+u))/ -
2g Ioe du du -9/2--
(52)
Vu
The exact result involves the paraholic cylinder function:
(gg) = v e g/4 D-1/2
(53)
where D2(2) is the parabolic cylinder function.
D.2 ASYMPTOTIC EXPANSION
Steepest Descent Analysis The critical point of V(x) = x² + x is at xo = with V(0) = 0. Near this point:
V(x) = x² x1 (1+x
(54)
Expanding e =x/19 in the Gaussian measure e -2/1
Z(g)
dr x² 1
(55)
00
dr e -
(56)
klgs
k=0
dr ck=2//
(57)
klgs
A
-00
Using the Gaussian moment formula:
dx x 2n 22/9
/Tg
(2n - 1)!!
g" //g
[(n + 1/2)
9"
(58)
r(1/2)
This gives the asymptotic series:
Z(g) ~ V ang
(59)
k=
16

---

Published as a conference paper at ICLR 2026
where the coefficients are:
OA = (-1)* (4k)1 (k(k)
(-1) r + 1)
(60)
T(A: + 1)24
The first few coefficients are:
ag
(61)
3 -0.375
d1
(62)
105
0.8203125
(63)
128
10395
03
=
10. 1517117777
(64)
1024
2027025
a4
247.4415283208125
(65)
8192
D.3 BOREL TRANSFORM CONSTRUCTION
Definition and Computation
The Borel transform of the asymptotic series is:
B[Z]({) Σ
ak
ck
ak
(66)
T(k 1)
k!
Substituting our coefficients:
00
(-1)* (4k))
(41))
([Z
(67)
k!
4k (kd))²
(kt)3
Jem
Am
This can expressed terms of hypergeometric functions:
BiZ](<) oF3 ( 1 1 3 256
(68)
2''2
27
Singularity Analysis The hypergeometric function F.3 has singularities when its argument ap- proaches values where the series diverges. For our case, the dominant singularities occur at:
256ÇI
1mck
k 1,2,3
(69)
27
3
This gives the singularity locations:
Ca 256
27
2mm
22/3
1//
k4
π
2
2/3
k4
(70)
3
3
3
However, the exact analysis shows the simpler result:
2 2//3 b=
2
25/3
S+
k, 4: 1,2,3....
(71)
3
The dominant singularity is:
25/3
2
C1
R3 1.0578
(72)
3
3
17

---

Published as a conference paper at ICLR 2026
D.4 STOKES PHENOMENA AND LATERAL RESUMMATION
Laplace Transform and Ambiguity transform:
To recover the original function, we apply the Laplace
Z(g) -[
4/s ((Z)(tt
(73)
9
However, the integration path passes through the singularity at t = (1. creating an ambiguity. We must deform the contour above below the real axis:
race
Z+(0)
-
1/9 (t)
(74)
9 Jo
Computing the Discontinuity
The discontinuity across the branch cut is given by:
Z+( (g)
2Ti
Resi=Ca (e (t))
(75)
For the dominant singularity at (1. the residue calculation gives:
Rest= [e-v/B[Z](()] = Age Sa/9
(76)
where A₁ is the Stokes constant.
Near t = S1, the Borel transform behaves as:
A
(t) ~ (t 1/2 + regular terms
(77)
This gives:
A1 = lim (t B(ZT(()
(78)
t-
From the hypergeometric analysis:
A 2/T 31/4
R 2.128
(79)
D.5 TRANS-SERIES CONSTRUCTION
Non-perturbative Sectors bative and non-perturbative contributions:
The complete solution involves a trans-series that includes both pertur-
Z(g) ( (g) +
(80)
Arm
where:
( = Tg
ang
(perturbative)
(81)
n=0
Do
zik AgE
kC1/9
Tg
an
g"
(non-perturbative)
(82)
n=0
Computing Von-perturbative Coefficients are determined by the alien derivative structure. For the first non pertururueeeeve sector:
The coefficients in the non non- nent-etuurbatve sectors
(1) A a0p
(83)
5
where A is the alien derivative at Ç1.
The alien derivative satisfies:
n -
! 10n
am 1-m
(84)
m==0
This gives recursive structure linking all sectors of the trans-series.
18

---

Published as a conference paper at ICLR 2026
D.6 RESUMMATION AND RECOVERY
Borel-Padé Resummation To implement the resummation numerically, Borel-Padé approx- imants. Given the asymptotic series coefficients {ax NN k= we construct:
(c)(()
(C)
(85)
Qx(()
where Pas and QN are polynomials chosen to match the first M + terms of the Borel transform. The resummed function is then:
{
(
(86)
Jo
Numerical Implementation
For practical computation, we use the following algorithm:
Algorithm Borel Resummation of Quartic Oscillator
Compute asymptotic coefficients ag for 0, 1...., N
2: Construct Borel transform coefficients by = 02 kk!
3: Build Padé approximant B M from {b&}
4: Integrate: (g) 1 % e /
5: Add non-perturbatiiv corrections: Z (g) Z(M/NN(g) Aie A 4/3z(M//1
(g)
D.7 VERIFICATION AGAINST EXACT SOLUTION
Numerical Comparison
We can verify our resurgence analysis by comparing with the exact For small g:
solution equation 53
g
Exact Z(g)
Asymptotic (5 terms)
Borel-Padé [2/3]
Full Trans-series
0.1
1.7724
1.7023
1,7721
1.7724
0.05
1.2533
1.1584
1.2531
1.2533
0.01
0.5606
0.4524
0.5605
0.5606
0.005
0.3960
0.2883
0.3959
0.3960
Table I: Comparison of different approximation methods
Error Analysis
The error in the asymptotic series truncated at optimal order Nopt R CI/g is:
[gg) Z No. (g)) Si/9 e -1.588/g
(87)
The Borel resummation reduces this error exponentially, while the full trans-series achieves machine precision accuracy.
E TRAINING LOSS VISUALIZATIONS
Here we attach the visualizations from the result section 4
We also include the evaluation loss curve when training small transformer on standard Shakespeare dataset with a randomly chosen critical target values as an ablation test on the usefulness of the targets computed by SURGE. The light red curves surrounding the bright red dotted line indicates the uncertainty given by different seeds. We see that in figure [5] random targets with different seeds do not accelerate training well as the SURGE computed targets.
19

---

Published as a conference paper at ICLR 2026
SIMAt kegel Ho
50
sunce Sroth
137
500 Mypoemaooo
Alat Apoientter
SUNICE Appren mutonn
AlamhWiounE Comper son
-13
THE
Sance
Mickeey
Preditionn Analyn
2U122 Adden CEleratio
300
20
20
10
10
130
800
120
Figure 2: 1d regression with 2 layered-MLP
Table 2: Validation Loss Comparison: SURGE vs Learning Rate Scheduling Strategies across Different Initial Learning Rates (200 epochs, MNIST local minima architecture)
Method
LR=1e-4
LR=3e-4
LR=le-3
LR=3e-3
LR=1e-2
LR=3e-2
Constant
1.4079
1.4103
1.1417
1.2924
1,4798
1,9418
StepL.R
1.6747
1.4129
1.3549
1,1810
1.4047
2.2991
Exponentiall.K
2.1761
1.6701
1,3062
1.2308
2.2995
1.5639
CosineAnnealing
1.4259
1.3593
1,3010
1.6238
ReduceOnPlltcaa
1.4864
1.4628
1,3185
1.2272
1.2215
1.9176
SURGE
1.4284
1,3918
1.2617
0,9913
1.1127
1.6823
Best Scheduder
1.4079 (Const)
1.4103 (Const)
1.1417 (Const)
1.1810 (Step)
1.2215 (Plateau)
1.5639 (Exp)
SURGE Improvement
+1.5%
+1.3%
-10.5%
+16.1%
+8.9%
+7.5%
F COMPARISON WITH STANDARD SCHEDULERS
In this section, we summerize the comparison of SURGE with standard schedulers on Adam on MNIST using a standard MI.P network as in the main text. We further And we take the loss curve from learning rate 0.003 and 0.0003 for comparison.
We also add experiments testing overfitting in the 1d function fitting case. Running the same function fitting optimization multiple times with different random states, we have the following test vs train mean square error comparison. We see that SURGE induces overfitting compared to the original optimizer.
20

---

Published as a conference paper at ICLR 2026
Learning
AdamW All Learning Rates
22"
SURCER DA
13
wmm
14w237
12 100
12
Epach
100
(a) Adam optimizer
(b) Adam optimizer
SGD All Learning Rales
Muon All Learning Rates
2 10
400 tedd
BuR0S 14:00
SURICC 24-30
12
Unkiortee
10x10'
14x 10
14
12"20
20
spachh
(c) SGD optimizer
(d) Muon optimizer
Figure 3: MLP training on standard MNIST classification task.
21

---

Published as a conference paper at ICLR 2026
Learningg
AdamiW All Learning Rates
1x
0802
4x
Tec3
SUROE der
Wildation L444
34
2.6 307
200
200
3000
Tyach
(a) Adam optimizer
(b) Adam W optimizer
Learning
33"
38
Valdatten Lo00
24
32x
Epech
330
/10
300
(c) SGD optimizer
(d) Muon optimizer
Figure 4: Small transformer training on standard Shakespeare dataset.
40
55
3.6
3.00
220
302
350
200
200
200
400
600
Epach
(a) learning rate 0.001
(b) learning rate 0.002
Figure 5: Ablation test using randomly generated targets for small transformer training on standard Shakespeare dataset using Adam.
22

---

Published as a conference paper at ICLR 2026
LR 3e-03
LR 3e-04
Consuat
2x
22x
2x
Conteraned
Validation Losa
Valitation Loss
10"
1.4
25
100
125
175
200
25
175
200
Epoch
Epoch
(a) Adam with MNIST
(b) Adam with transformer
Figure 6: Comparison between SURGE and learning rate schedulers
Train vs Validation Loss
2,00
1.75
1.50
1.25
Validation MSE
1.00
0.75
0.25
Adem SURGE
1.00
1.25
1.75
Train MSE
Figure 7: Generalization comparison between Adam 1d function fitting and SURGE Adam.
23


Reviews:
[{"review_id": "DmaBhEwSaK", "reviewer": "Reviewer_rhPe", "summary": "This paper introduces SURGE, a novel optimization framework that utilizes resurgence theory to uncover global structural information encoded within divergent asymptotic series. In particular, the factorially divergent perturbative expansions of parameter-space partition functions intrinsically capture all critical objective function values through the singularities of their Borel transforms. Algorithmically, SURGE operates by constructing the partition function of the loss landscape, extracting its asymptotic expansion coefficients, and identifying singularities in the Borel plane that correspond one-to-one with critical objective values. These values serve as global targets, offering principled guidance for local gradient-based optimizers via adaptive learning rate modulation. Empirically, SURGE demonstrates consistent performance gains of 15–30% across diverse settings, ranging from function approximation to large-scale neural network optimization (Transformer).", "questions": "- Could the authors provide a computational complexity analysis of SURGE? In particular, is the proposed framework scalable to modern large-scale architectures (e.g., models with millions or even billions of parameters)?\n\n- Can the authors include an additional experiment to empirically verify that the detected Borel singularities indeed correspond to all critical points of the objective function?\n\n- In Figure 3c, SURGE appears unstable and performs worse than SGD with a learning rate of 1e-3. Could the authors provide an explanation or analysis of this behavior?\n\nMinor points:\n\n- Line 345: The reference to “2” should be replaced with “Algorithm 2.”\n\n- Section 4.1 would benefit from a brief paragraph summarizing and interpreting the experimental results of SURGE.\n\n- Figure 1’s legend appears incomplete, with eight curves but only six labels. Please revise to ensure consistency.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper establishes a rigorous theoretical correspondence between Borel singularities and critical points of the objective function, thereby providing a principled mechanism to extract global geometric information from the optimization landscape. This contribution is particularly significant, as most existing optimization algorithms (e.g., SGD, Adam) are inherently local and lack awareness of the broader landscape structure.\n\n- Empirically, SURGE demonstrates consistent improvements of 15–30% over conventional optimization approaches across a variety of problem settings, highlighting both the robustness and practical potential of the proposed method.\n\n- The paper is well-written and self-contained.", "weaknesses": "- The partition function integral $Z(g)$ is computationally intractable in high-dimensional parameter spaces. Approximating it with a learned sampler $q_\\psi(\\theta|g)$ introduces an additional optimization layer, which may incur significant computational cost and potential instability during training.\n\n- The experimental evaluation is relatively limited in scope, focusing primarily on small-scale setups—including synthetic function approximation, a modest MLP on MNIST, and a lightweight Transformer with only 10k parameters on Shakespeare text. Validation on larger and more realistic architectures would strengthen the empirical credibility of the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AycgCAMKEb", "reviewer": "Reviewer_YZ6g", "summary": "The authors propose a boost to local searching algorithms for objective functions that is able to aim critical points by exploiting global (although approximated) information about the function itself. They show how this improves time of convergence in a few examples.", "questions": "There must be limitations to this approach otherwise we should believe that it can make GD able to solve NP hard problems in polynomial time which it would be the astonishing main result, were it true, but it is not mentioned, therefore there must be something I do not understand. The authors should discuss how NP problems remain not solvable in polynomial time even with this approach and in general discuss limitations more explicitly, beside the advantages.\n\nFor rough objective functions the evaluation of the partition function should be exponentially hard. What is the cost expected also considering the approximation proposed?\n\nSGD does not necessarily see the same landscape as the objective function. \n\nHitting critical points is not necessarily useful, especially when there is a huge number of saddles and maxima, and few minima which may be even fewer if associated to low values of the objective function. In fact it would be better if GD could avoid saddles, rather than hitting them, as they slow the dynamics down without giving other advantages. How can the algorithm select among the critical points? \n\nIt was appreciated the intuitive introduction to resurgence. It is way less clear the implementation of the algorithm. Although apparently explained in multiple places, I cannot find intuition on how it must work.\n\nLine 170-171 “we see where it backfires at us in the example” which example? It should be made clearer.\n\nIn figure 1 on both sides there are 4 double series of data, while legends refers to only three choices of parameters. In some instances long time behaviour between blue and red is very similar, in other very different. What is the reason for that?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "It seems a truly original and intriguing piece of work, based on robust mathematical theory.\nIt is visible the effort towards clarity and very appreciated, however it remains mysterious the intuition about the algorithm proposed.", "weaknesses": "It is visible the effort towards clarity and very appreciated, however it remains mysterious the intuition about the algorithm proposed.\nIt is also missing a critical discussion on the limitations of this approach, despite the pretended global information obtained about the objective function. It cannot be the full information otherwise it would be overcoming NP hardness.\nWithout these discussions it is hard to understand what is the actual contribution to improve algorithms convergence in the general case, this explains my conservative rating FAIR to “contribution”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PriSkksjcE", "reviewer": "Reviewer_Zpez", "summary": "The paper proposes SURGE (Singularity Unified Resurgent Gradient Enhancement), a novel optimization framework that leverages resurgence theory (via Borel transforms of partition functions over neural-network parameter space) to identify critical objective values in the loss landscape. These values are used to guide adaptive learning rates for any base optimizer, with the aim of improving convergence and escape from sub-optimal local minima. The authors provide theoretical arguments linking Borel singularities of the network partition function to landscape critical values, and demonstrate empirical improvements (e.g., Sec. 4.2, ~10K-parameter networks) compared to baseline optimizers. \n\nThe approach is conceptually intriguing and mathematically elegant, and has potential to open a new line of optimization research. However, I feel that the current version may fall short in practical guidance, sensitivity to numerical computations, scalability evidence, and empirical benchmarking.", "questions": "(1) In addition, I wonder how the quality of numerical approximation of asymptotic series, equation (29), affect the overall performance of adapting learning step when employing (33) and (34).\n\n(2) The paper reports improvement in final objective values, but lacks wall-clock time comparisons against standard optimizers.\n\n(3) I wonder sensitivity analysis and performance under noisy given data of the proposed algorithm as a wrapper for general optimizer with adaptive learning rate.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "(1) From my point of view, the use of resurgence theory and partition-function singularity detection to gain global landscape information is highly innovative in ML optimization work.\n(2) The algorithmic wrapper (SURGE) is optimizer-agnostic and shows promising empirical results.\n(3) The connection between complex-analysis tools (Borel transforms) and ML optimization adds theoretical depth and opens a new direction of research.", "weaknesses": "(1) \tThe method hinges on computing a partition function and identifying singularities to yield target values; however, the manuscript lacks guidance on how to choose the initial model parameters or coupling parameter g in general settings (even simple losses like cross-entropy or MSE). The authors should provide practical heuristics or guidelines for parameter initialization. \n(2) \tComputing the partition function Z(g) over a coupling range and performing Borel analysis seem to be likely very expensive and may restrict the method’s applicability. I feel that the method may simply shift the optimization burden onto a heavy pre-computation. More discussion is needed on the cost and scalability trade-offs. In addition, I wonder how the quality of numerical approximation of asymptotic series, equation (29), affect the overall performance of adapting learning step when employing (33) and (34).\n(3)  Although the method is theoretically rich, the empirical setting (10K parameters) is still modest relative to real-world deep networks (millions of parameters).  Also, the authors note that SURGE may amplify over-fitting by pushing harder toward identified critical values, but no empirical analysis or mitigation strategy is provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cxBVCRzZpi", "reviewer": "Reviewer_fviL", "summary": "This work proposes to inject non-local informtaion about the objective function into the optimization process.\nIt relies on resurgenece theory to obtain the function values of the critical points of the objective function and then uses this information for learning rate adaptation.", "questions": "Please address \"Lack of connection between \"2.1 An intuitive intro to resurgence\" and the rest of the paper\", \"Unclear what is new vs existing and missing references\" and \"General lack of literature review\" above.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "As far as I can tell, the use of resurgence theory to improve optimization algorithms is novel and this idea seems to have a lot of potential. If one can indeed obtain a god approximation function values of the critical points of the objective function, this could dramatically improve optimization algorithms.\n\n\nThis work provides a thorough introduction on this topic.", "weaknesses": "### Lack of connection between \"2.1 An intuitive intro to resurgence\" and the rest of the paper\nThis work starts with a lengthy introducion to resurgence theory. However, it is not clear how large parts of this introduction relate to the rest of the paper. As far as I understand, the discussion about the integration contour 172-205 is not necessary to understand the rest of the paper and only confuses the reader.\n\n### Unclear what is new vs existing and missing references\n- There should be a reference for Thm. 1\n- I don't understand whether Prop. 2 and Thm. 3 are novel or existing results. If they are existing, please provide references. Also, both the statement and the proof of Thm. 3 need much more details: how is the integration over the level set done for instance?\n\n### General lack of literature review\nOptimization on non-convex landscapes as well as adaptive learning rate schemes are both very active research areas but the discussion of related work is very limited.\nIn particular, the following topics should be discussed:\n- Adaptive and parameter-free leanring rates, with a particular focus Polyak-type schemes, which also use function values to adapt the learning rate.\n- Langevin dynamics on non-convex landscapes: these methods sample from the Gibbs distribution, which is closely related to the statistical mechanics discussion in Sec. 2.3.\n- Simulated annealing and tempering methods, which also rely on the partition function to explore non-convex landscapes and therefore seem related to the proposed approach.\n\n### Unclear how the practical algorithm follows from the theory\nMost of the steps of the algorithm in 3.1 appear pretty ad-hoc and it is not clear why the authors chose these particular steps.\n- In particular, at l32, why choose the maximum value in T lower than the current function value? Why not the minimum? Indeed, if the theory is correct, the set T contains the function values of all critical points: therefore one should only choose the minimum one, since it corresponds to the global minimum of the objective function. This discrepancy between theory and practice is never discussed.\n- Why use this function value to adapt the learning rate as in l34? Why not use it in other ways? In particular, why not use a Polyak-type learning rate scheme?\n\n### Experimental section is weak\nAll experiments are on toy problems (1D function or MNIST), hyperparameters are not proeprly tuned (one should report the result for the best learning rate for each method), there are no error bars, no adaptive learning rate baselines (e.g., Polyak-type schemes at the very least) and no clear benefit of the proposed method is shown.\n\nMoreover, it is not clear whether the resurgence theory is indeed useful for the proposed algorithm: one could imagine much simpler approaches.\nFor instance at L342, this work explains how to obtain an approximation of the partition function \n$$\nZ(g) = \\int e^{-f(\\theta)/g} d\\theta\n$$\nwhich I agree is an interesting idea. However, the authors then use the intricate machinery to exract the function values of the critical points from this partition function while, if I am not mistaken, $- g \\log Z(g)$ for small $g$ is already a good approximation of the global minimum of $f$. This value could then be used in a Polyak-type learning rate scheme or in the proposed learning rate adaptation scheme. It would be interesting to see whether this simpler approach, that does not use resurgence theory, works just as well as the proposed method.\n\n### Not clear how the proposed scheme helps escaping \"suboptimal regions\"\nThe abstract states that \"These target values provide global guidance to local optimizers, enabling [...] escape from suboptimal regions\". However, it is not clear how the proposed learning rate adaptation scheme helps escaping suboptimal regions. If the function value of a critical point is used to adapt the learning rate, and if the current iterate is in a basin of attraction of a suboptimal local minimum, then the adapted learning rate will likely be small (since the function value at the current iterate is close to that of the suboptimal local minimum). Therefore, it is not clear how this scheme helps escaping suboptimal regions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)