Convergence of Muon with Newton-Schulz
Authors: Gyu Yeol Kim
Min-hwan Oh
Venue: ICLR 2026 Poster (2026)
Keywords: Muon
Newton–Schulz
Orthogonalization
Nonconvex Optimization
Primary area: optimization
TL;DR: Muon with Newton–Schulz as used in practice converges at the SVD-polar rate up to a constant that shrinks doubly-exponentially, which explains why using 2–3 steps is both fast and accurate and improves rank dependence over SGD with momentum.

Abstract: We analyze Muon as originally proposed, using the momentum orthogonalization with a few Newton-Schulz steps. The prior theoretical results replace this key step in Muon with an exact SVD-based polar factor. We prove that Muon with Newton-Schulz converges to a stationary point with the same rate as the SVD-polar idealization, up to a constant factor for given the number of Newton-Schulz steps $q$. We further analyze this constant factor, and prove that it converges to 1 doubly exponentially in $q$ and improves with the degree of a polynomial used in Newton-Schulz for approximating the orthogonalization direction.
We also prove that Muon improves the rank dependence compared to its vector-based counterpart, SGD with momentum. 
Our results explain why Muon with a few low-degree Newton-Schulz steps matches exact-polar (SVD) behavior at much faster wall-clock time, and explain how much momentum matrix orthogonalization via Newton-Schulz benefits over the vector-based optimizer. Overall, our theory justifies the practical Newton-Schulz design of Muon, narrowing its practice–theory gap.

Published as a conference paper at ICLR 2026
CONVERGENCE OF MUON WITH NEWTON-SCHULZ
Gyu Yeol Kim
Min-hwan Oh
Scoul National University Seoul, South Korea
Seoul National University Seoul, South Korea
gyuyeolkim@snu.com
minoh snu. ac. kr
ABSTRACT
We analyze MUON originally proposed and used in practice-using the mo- mentum orthogonalization with a few NEWTON-SCHULE steps. The prior the- oretical results replace this key step in MUON with exact SVD-based polar factor. We prove that MUON with NEWTON-SCHULZ converges to a station- ary point the same rate as the SVD-polar idealization, up a constant fac- tor for given number q of NEWTON-SCHULZ steps. We further analyze this constant factor and prove that it converges improves with the degree of the polynomial used in NEWTON- NEWTON-SCHULZ for ap- proximating the orthogonalization direction. We also prove that MUON improves the rank dependence compared to its vector- based counterpart, SGD with momen- tum. Our results explain why MUON with few low-degree NEWTON-SCIIU steps matches exact-polar (SVD) behavior at much faster wall-clock time and explain how much momentum matrix orthogonalization via NEWTON- SSHUULZ benefits over the vector- based optimizer. Overall, our theory justifies the practical NEWTON-SCHUI.7 design of MCON, narrowing its practice-theory gap.
doubly exponentially in q and
INTRODUCTION
Modern deep neural networks comprise billions of parameters and demand highly efficient training procedures. A persistent challenge is that most widely used optimizers-such stochastic gradi- ent descent (SGD) (Robbins & Monro, 195T) and adaptive methods such as Adam (Kingma & Ba 2015 operate on vectorized parameters, thereby discarding the native matrix structure present in linear layers and attention projections. Optimizers that explicitly respect matrix structure can, in principle, yield search directions that are better aligned with the underlying geometry while remain- ing computationally efficient at scale.
2024) is an optimizer designed for matrix-structured parameters. At each iteration, instead of following the raw momentum, MUON orthogonalizes the momentum matrix and then uses this orthogonalized direction to update the weights. practice, this orthogonal- ization is not computed via an exact singular value decomposition (VVD)wwhich is accurate but expensive-but is approximated efficiently by a small, fixed number of NEWTON-SCHUI steps. Empirical studies (Jordan et al.) 2024| Liu et al. 2025a) have reported strong performance at scale with this SVD-free implementation, making MUON an attractive alternative to vector-based opti- mizers. Despite recent attempts analyze the convergence of MUON (Shen et al.) 2025 Li &
MUON (Jordan
Hong 2025; Sato et al., 22225) Pethick et al. 2025a b), theory still lags behind practice. Exist- ing analyses typically study idealized variant that replaces the NEWTON-SCHULZ step-central to practical MuoN-with an exact polar step computed by SVD for analytical convenience. This leaves open whether the actual SVD-free orthogonalization used in practice-i.e. a finite number of NEWTON-SCHULZ steps-addme NEWTON NEWTON-SCHUI. approximation impacts rank dependence and efficiency. Therefore, the follow- ing research questions remain open:
principled nonconvex convergence guarantees, and how the
Research questions.
Does MUON with NEWTON- HEWTON-SCHULZZ admit nonconvex convergence guarantees, and how do its rates compare to the exact SVD-polar idealization?

---

Published as a conference paper at ICLR 2026
How does the NEWTON-SCHULZ steps q and control the cesuracc-compuue trade-off? In particular, how large is the gap caused by using NEWTON-SCHULZ. and how does that gap become negligible?
polynomial degree
Can we show that MUON converges faster than the vector-counterpart, SGD with momen- tum? What geometric mechanisms and rank dependence drive this gap?
To address these questions, we analyze MUON as originally proposed (Jordan et al. 2024 and as used in practice: MUON with momentum orthogonalization computed via NEWTON SCHULZ. For nonconvex objectives, under standard smoothness assumptions, we prove the convergence of MUON stationary point, measured by the nuclear norm of the gradient. We establish that the conver- gence rate in the number of iterations matches the idealized (but not used in practice) -VD-based exact polar variant, up to constant factor that depends on the polar approximation error Eg (defined in Definition 3) for the fixed number of NEWTON-SCHUT, steps q. Moreover, we show that the po- lar approximation error Ey shrinks doubly exponentially as q grows and decays with larger K, which is the degree of the polynomial used in NEWTON-SCHULL steps. Recursive updates using this poly- nomial allow the optimizer to find the approximated orthogonalization direction of the momentum matrix. Hence, with only few NEWTON-SCHULD steps, the convergence rate of MUON quickly tends to the convergence rate of the exact polar step via SVD. Consequently, our results imply that, because a few NEWTON SCHULL steps far cheaper per iteration than SVD, the practical MUON implementation with NEWTON-SCHUL attains substantially faster wall-clock convergence.
Our main contributions summarized as follows:
The first convergence result of MUON with NEWTON-SCHULZ. To knowledge, we present the first nonconvex convergence guarantees for MUON with a finite number of NEWTON-SCHULE steps (Theorem [ originally proposed and practically used. It is important to note that even for convex optimization, the convergence of MUON with NEWTON-SCIULLZ has not been shown previously. The key distinction from the existing analyses of MUON is that we do not replace NEWTON-SCHULZ in the original MUON with the exact polar computed by SVD.
Analysis of polar approximation error and wall-clock convergence. We prove that the polar approximation error Eg due to using NEWTON-SCHULZ instead of SVD, in the MUON optimizer, decays doubly exponentially with the number of NEWTON-SCHLI steps and decays with the degree of the polynomial required to approximate the thogonalization direction of the momentum matrix (Definition 2). Thus, even with a few steps of NEWTON-SCHIU arbitrarily close to that of the SVD-varian in the number iterations (Theorems and 4 Hence, given that per- per-iteration computation is much more efficient for NEWTON- SCHULZ steps compared to SVD, the overall convergence in wall-clock time is much faster for MUON with NEWTON-SCHU.Z.
the convergence of MUON with NEWTON-SCHIULZ becomes
Sharper rank dependence in MUON with NEWTON-SCHULZ To prove the compara- tive advantage of MUON against the vector-based counterpart, we demonstrate for the first sharpens the rank dependence of the momentum
time that MUON with matrix in the leading term, compared to its vector-based counterpart, SGD with momen- tum. (see Table and Theorem 3).
2 RELATED WORK
MUON and momentum orthogonalization Jordan et al. (2024) introduced MUON, which orthog- onalizes a momentum matrix via a few NEWTON-
steps (SVD-free), and reported strong empirical results at LLM scale (Liu et al. 2025a). Earlier work orthogonalized gradients by SVD before applying momentum (Orthogonal- SGDM: Tuddenham et al. (2022)), whereas MUON ap- plies momentum before orthogonalization and replaces SVD with NEWTON- NEWTON-SCHULZ, resulting in faster computation with only matrix multiplications. More details about MUON are described in 333
MUON. Matrix-aware optimizers such SHAMPOO (Gupta al. 2018) SOAP (Vyas et al. 2024) and their variants (An et al.) (025) are second-ordas precon- ditioners: they maintain layerwise curvature Kronecker-factored second moments) and periodically
Second-order preconditioners
2

---

Published as a conference paper at ICLR 2026
apply inverse-root preconditioning. By contrast, Muon is not a second-order method; it neither timates nor inverts curvature but orthogonalizes the momentum matrix via a few Newton-Schuld iterations (SVD-free, using only matrix multiplications). These methods target different mecha- nurmt--uuvvature preconditioning vs. projectionnommalizzionn-aaa can be complementary rather than directly comparable.
Practical efficiency of MUON. Large-scale training reports for MUON (Liu et
al. 2025a Shah et al.
2025| Tveit et al.) 2025) and cotmmmniiaaiio/mmmooyyyaeoo
variants (Ahn & Dion [025 | Liu al.,
2025b) motivate a theory that is SVD-free and GPU-aligned. Our analysis of NEWTON-SCHUT key step the MUON optimizer, adopts precisely that stance. The analysis provided in this work can be adapted to other variants of MUON.
Convergence analysis of MUON. Several recent analyses examine MUON but typically either ide- alize the orthogonalization by assuming an exact SVD polar step (Shen et al. 2025) work un- der
with dimension-driven constants (Li & Hong 2025) focus on stabil- ity/variant phenomena (Sato et al.) 22025) or offer complementary lenses (steepest descent under norms, trust-region views, implicit constraints, or LMO/Frank- Wolfe formulations) without address-
ing NEWTON-SCHULZ step accuracy in nonconvex rates (Bernttei & Newhouse, 2024 Kovalev
2025
Chen et al. 2025 Riabinin et al., 2025; Pethick et al. (025a
Key distinctions compared to the existing analyses of MUON. Prior work either assumes exact SVD polar steps, measures progress in a geometry that obscures rank benefits, or does not quantify the approximation from NEWTON-SCHULAS in MCON. Our work is the first to analyze how two key parameters- the number of NEWTON- NEWTON-SCHULE steps and the degree of the polynomial used for finding the orthogonalization direction of the momentum matrix approximately- vergence rate of MUON. The results indicate a nonconvex convergence rate to stationary point, an explicit and rapidly vanishing constant factor derived from NEWTON-ScHUL and sharper rank dependence than SGD with momentum under the same metric. Overall, our work explains why MUON converges quickly (particularly wall-clock time) as well as why only a few steps of NEWTON-SCHULL suffice in practice.
the con-
instead of SVD,
3 PRELIMINARIES
3.1 NOTATIONS
For matrix X m X is its transpose. For X = (are)) E R.xx, tr(X): N.. We write || || and || for the nuclear, spectral (operator), and Frobenius norms, respectively, and (X, F) == tr(XTY) For a thin SVD X = UEVT, the polar factor is Polar(X) := UVT, a partial isometry with || |Plla((X X) llop I and (X, Polar(X)) = ||X||- (See Appendix A.I For
two sequences {an and (bn n==1 an O(b,,) implies that there exists a constant C >> 0 such
that an S Cb, holds for all n 2 1. We use B[-] for expectations over all algorithmic randomness.
3.2 PROBLEM SETTING: NONCONVEX OPTIMIZATION
We consider the stochastic optimization of a matrix parameter: W E Rm min
W W S(W) EE ());
where the objective o is nonconvex with f == inf f(W) > -O. We denote by r == min min(m, n) the maximal possible rank of the matrix. At iteration f, batch & (Et,1,..., Et,B) is drawn with {E J i.i.d., and {&r}e are independent across l = 1,...,T. T. At t = O, the model parameter
Wo E I is initialized, and we define the initial sub-optimality as D = S(W)) - f³. In this paper, the following assumptions are made for the convergence analysis: Assumption 1 (Lipschitz smoothness). The objective function f Ruxx + R is continuously differentiable and L-Lipschitz smooth, i.e., all X,Y E RRx,
[V (((II I -IX--Y -llop-
We use smoothness with respect to the operator norm (and the nuclear norm as its dual). Assumption 2 (Bounded variance). We assume V { is an unbiased stochastic estimator of the true gradient V fW) and has bounded variance for all W and single sample &, i.e.
E V f(W: v((W), ((W - vf(W)]l?) < so
3

---

Published as a conference paper at ICLR 2026
For mini-batch of size B, the variance is at most o²
Assumptions] 1 and are standard for analyzing first-order methods in stochastic optimization (Boyd
& Vandenberghe, 2004 Nemirovski et al., 2009) Candes & Recht, [ [Shairo et al. 2021] Reddi al. 2018 Zou et al. 2019). L-smoothness typically defined using norm and its dual, and the specific operator-nuclear geometry chosen here is a natural fit when working with matrix parameters
2004
and polar orthogonal updates (Nesterov 2013 Beck 2017: Jagei 2013). The bounded-variam assumption, where the mini-batch variance is o² B, is a foundational concept for algorithms like
SGD in both convex and nonconvex scenarios (Bottou et al., 2018 Ghadimi & Lan 2013).
In this paper, the metric for the convergence rate is defined as follows:
Definition 1 (e-stationary point). We call W Rmxx e-stationary point (in the nuclear norm) if (W]ll & e. Equivalently, we say an algorithm attains e-stationarity in T steps if
T
Note that when working with functions that have matrix inputs, using the nuclear norm find stationary point provides a more restrictive and precise condition than using the standard Frobe- nius norm. In other words, if point satisfies the stationarity condition for the nuclear norm, it is guaranteed to also satisfy the condition for the Frobenius norm.
3.3 MUON ALGORITIIM AND NEWTON-SCHULL ORTHIOCONALIZATION
Algorithm MUON (with the illustration of NEWTON-SCHULLE orthogonalization)
Require: Learning rate 7 > O, momentum 3 E (0,1), batch size B, NEWTON-SCHUL. polynomial (degree k), total iteration T.
steps g e
1: Initialize: Mg 0, Wo R
2: for t = to do
3: G, _
Σ -i=1 (W(
Compute (batch) gradients
4: Mr + 3M + G:
5:
X1.0 M1/a; with a; [|
> Pre-NEWTON-SCHULZ scalin
6:
for to do
7:
Xt.j f Pm (X X. - X1.5-
NEWTON-SCHULZ steps (Lines 6-9)
8:
end for
9:
O2 - 1
10:
W, + Wi-1 7
Update parameters
11: end for
For clarity of exposition, we present pseudocode for MUON with explicit illustration of the NEWTON-SCHULZ steps (Lines 5-9) in Algorithm [ Note that this is not a new algorithm; the original method of Jordan et al. (2024). here written in more general mini-batch form with a step-by-step illustration of NEWTON- SCHULZ. Rather than orthogonalizing via SVD, MUON ap- proximates the orthogonalization direction using only matrix multipicaations; the key mechanism enabling this is the
SCHuLzzbased orthogonalization.
The key advantages of using NEWTON-See SSHNNNSSSZZZZA for orthogonalizztton are follows:
NEWTON-SCHUL.2 makes MUON inversion-free and SVD-free. SVD is computation- ally expensive and makes each iteration costly. In contrast, the NEWTON-SCHULZ ap- proach relies solely on matrix multiplications, yielding substantially better per-iteration essicien for large parameter matrices.
NEWTON-SCHUS is an iterative method that allows for precise control over the degree of orthogonality by adjusting the number of iterations. The number of TEWTON-SCHULE steps provides a direct trade-off between computational cost and the degree of orthogonal- ity, offering valuable flexibility.
4

---

Published as a conference paper at ICLR 2026
3.4 NEWTON- SCHULZ POLYNOMIAL
In the MUON algorithm, at every iteration, a scaled momentum matrix X is orthogonalized via NEWTON-SCHULAS steps. First, the matrix X is formed and is then passed to a polynomial function with degree K. Recursive updates by this polynomial make the matrix X nearly orthog- onal, i.c. XX T I. We first define this function px in Definition 2
and state the properties of this
polynomial used in NEWTON-SCHULZ.
Definition (NEWTON- SEEWTON-SCHUL2 polynomial), For degree N, the NEWTON-SCHUL.2 polyyo-
mial is the Taylor truncation of 1/ v/vAal = i.e.
d dxa
(1)
x
1/2
A=1
for 1....... The explicit form of the NEWTON SCHUTLN---SCZ-Z-I polynomial for degree
(2s)
Pa(()) Σ -)-1- Cs
>
(s!)
Equivalently, with reparametrization
A [0, 1]. Px u) Σ C2
Proposition 1 (Properties of p.). For À E [0, 1]:
Positivity, Pa(A) and (A( 2 with equality iff X
Monotonicity of T. Let (A): = (An(A)]? then we have T non-decreasing on [0, 1] and T(1)
Consequently, for any symmetric A Z 0 with spectrum in [0, 1]. the NEWTON-SCHULZ update S 1: NEWTON-SCHIN SCHULZZ steps preserve unit spectral ball (see Appendix (). Moreover, the property of the function is used when proving how fast does one step of NEWTON ScHULZ make the momentum orthogonal (Lemma 2
+ Ap satisfies pi (p,
In order quantify the degree of orthogonality of the output matrix Xt.q after g steps of NEWTON-SCHUIZ and to measure the approximation error derived from NEWTON-SCHUT com- pared to the exact-polar method via SVD under operator-norm, define the following:
Definition 3 (Orthogonality residual and polar approximation error). For fixed t, let II, be the or- thogonal projector onto range(Mi). With { from Algorithm 7 residual degi and the polar approximation error εx,a by
define the orthogonality
81, II, - Xx, llop E (0, Ei.q := [Xeg Polar(Mi)loa
Define Eg = sup; Et.y and 8, = sup, 8e,0-
Remark 1. In MUON, there is scaling step for the momentum matrix before applying it to the re- cursive update by the NEWTON-SCHUT polynomial (Line Algorithm This scaling ensures || < 1, which is required to apply the NEWTON- SCHULZ polynomial properties described in Proposition [ In parallel, the initial residual is strictly less than 1, i.c., d,,0 [0, 1) for every iteration (see Appendix
4
MAIN RESULTS
We begin by stating our two main theorems: (i) nonconvex convergence rate for MUON with a finite number of NEWTON-SCHULLE steps (Theorem D: and (ii) explicit bound the multi- plicative constant induced by NEWTON-SCHULZ, together with its (doubly) exponential decay in q (Theorem 2}. We then provide brief proof sketches for both theorems in Sections 4.3 and 4.4 For comparisons, Section 4.5
4.5 also presents convergence rates for the idcalized MUON with an exact SVD-based polar step and for SGD with momentum the vector-based bascline- stated under the same nuclear-norm stationarity metric.
4.1 CONVERGENCE OF MUON (WITH NEWTON-SCHULA.
Theorem (Convergence of MUON with NEWTON CWVTON-SCHULZ), Suppose Assumptions Aadd holld and MUON (Algorithm) / with initialization Wo E Rmxn Choose the stepsize and momentum
5

---

Published as a conference paper at ICLR 2026
a) 1) V and = min
1 where r min(m, nJ. Then there exists a factor Xq > 0, depending only on the number g of NEWTON-SCHUL. steps, such that
VLDE VvTT
((rE)))
1/4
LD
T [WW S X
+ +
VVT+
VBT
Consequently, MUON with NEWTON- CEATTON-SCHLLZ...
actuizz.ttainss attains c-stationarity with an iteration complexity of
LD
T= max
iterations.
Bea
Be
Discussions of Theore] 1
that MUON with NEWTON- converges to an e-stationary point. To the best of our knowledge, this convergence guarantee is the first result for MUON with NEWTON SCHULZ. NEWTON-SCHULZ. Moreover, as shown later by comparison with the SVD-based
polar variant, the iteration complexity of MUON with NEWTON-Scin SCHULZ matches the exact-polat rate up to a multiplicative factor X, that depends on the polar-approximation error Eg (e.g., Table T Crucially, we show later in Theorem 2 that Xa + converges at an exponential rate in the number of NEWTON-SCHULZ steps q. so the convergence gap (in the number of iterations) to the ideal SVD-polar rate be made arbitrarily small. Since each NEWTON-SCHEE cheaper than an SVD, these results provide the first theoretical explanation for the superior practical performance observed for the original (SVD-free) MUON.
step is substantially
4.2 DECAY RATE OF Eg AND CONVERGENCE RATE X₃ ->1
We now quantify how fast X4 approaches 1. For a given number q of NEWTON-SCHUUS steps, we can show that the polar-approximation error Ey decays doubly exponentially (with faster decay for larger k). Since Xę is controlled by Eq. it follows that Xg + is at the same rate. The following theorem formalizes this result.
Theorem 2 (Upper-bounds on Ey and Xa). For the NEWTON -ONTON-SCHULZ polynomial with degree and for any t, 8t.g I 8(x+1)" Hence, the bound of the polar approximation error Eg and the factor Xg occurred by NEWTON-SCHULL is
60 (x++1) < 00
(x+1)²
Eq 5 1
Xq 1 Eq
60 1)
where 6g sup, 84,0
Discussion of Theorem The theorem shows that e, is bounded by 66** (x+1)) Hence, Eg vanishes doubly exponentially in q (and improves with larger k). Therefore, Xq + at the same doubly exponential rate. Together with Theorem [ this result implies that the iteration-complexity gap between NEWTON-SCHULZ and the idealized SVD-polar update becomes negligible after only few NEWTON-SCHUTL. steps.
Practical implication. A finite number of NEWTON- NEWTON-SCHULZ steps yields iteration complexity essentially indistinguishable from exact SVD updates (up the factor Xg doubly exponentially fast), while dramatically reducing per-iteration cost by using only matrix multiplications.
4.3 PROOF SKETCH FOR THEBRRMM
We briefly outline the main ideas. First, introduce the scaled momentum N₂ = (1-13) My (faithfully following the original update rule). so the EMA becomes N₂ +- 8N + (1 - B)G.. Next, apply descent lemma (Lemma 6) to the update W, +- W 1 which yields a term of the form (V Oz) Decompose this inner product as
(W) O.) (Ne, Ot) + (VS(W,-1) Nr, F.)r,
thereby isolating the momentum mismatch. Prior analyses typically stop here and average over iterations. In contrast, we further split (Ne, Ot) as
(Nr, O.) (N,, Pt) + (Ne, Oz Pt)r,

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of convergence rates.
Method
Convergence rate
AD BT
1/4
SGD with momentum (Theorem
3
1/4
MUON with SVD (Theorem 4
O
LD
VBT
MUON (Theorem
Xa O
10 BT
1/4
LD 7
/BT
((W) -'. = min m. J. iterations T, batch size B, Lipschitz constant L, variance bound The factor Xa converges at exponential rate step q. (Theorem 2 Xe S [1 161+1 where q the number of NEWTON-SCHULZ step and is the degree of the NEWTON- WANTON-SCHULLZ polynomial (Def. 2
separating the exact polar factor P: from the NEWTON-SCHULE orthogonalizer (the output matrix routine). As we define the polar approximation error Eg as the discrep- of the NEWTON-SCHUIE ancy between the exact polar factor P. Polar(N) = Polar M,) and the actual step O₂ produced by q steps NEWTON SCHULZ, NEWTON-SCHULZ, we can control this part with respect to Eg. This yields a one-step descent inequality for MUON that explicitly includes the NEWTON-SCHUL.-inne
error Eg. Av- eraging this inequality over / = I...., T and choosing and /3 as specified in Theorem produces
the stated convergence rate. Full details appear in Appendix
4.4 PROOF SKETCH FOR THEOREM
We outline how Theorem follows; full proofs appear in Appendix D Recall that in Theorem T the convergence rate contains the polar-approximation error Eg and the multiplicative factor Xa de- pending on Eq. To bound the resulting multiplicative factor Xq. we relate Ey to an orthogonality residual that quantifies how close the Concretely, letting Xt.g be the output matrix after 9 NEWTON-SCHUL2 steps applied to the (scaled) momentum at iteration f, define وτ,q == II,
COTOO-SCHILZ iterate having orthonormal columns.
XT, llop.
The next lemma provides the spectral link between the residual and the yolar-approximation error: Lemma (Orthogonality residual vs. Polar approximation error). Let X be the smallest positive eigenvalue of Xt.g X.gx, t. restricted to range(M, (sel A min = iaak (M,n = 0). Then
8i,g
- min'
Ft.g =
A = v 1
8r,g.
Next, we describe how a single NEWTON-SCHUIZZ step transforms the residual through the degree- polynomial px:
Lemma 2 (Residual update). For NEWTON-SCHUS SCHULZ polynomial px. the orthogonality residual dig is updated by NEWTON-SCHULL per step
8rjt1 = 0(ōr,))
where d(u) (1 u) (1 11 n.)
To prove Lemma 2 we introduce r(A) A (A)]2 note that 7 is non-decreasing on [0, 1] and satisfies r(1) < (Proposition]] D. and then translate this monotonicity to the residual map Φ.
Finally, we obtain a contraction bound and its multi-step consequence:
Lemma 3 (Residual decay by NEWTON-SCHULZ polynomial). For NEWTON-SCHULZ polyno- mial Pxo ((u) I u*+1 on [0, 1] where & is a function defined Lemma 2 Hence, for every t and all j 2 0,
{(++1)" 8000
8tjt1 S 6,55
degg
4.5 COMPARISONS WITH SGD WITH MOMENTUM AND MUON WITH SVD
To enable a fair comparison with the original MUON with establish convergence guarantees for baselines: (i) SGD with momenfum and (ii) the idealized
method, we also

---

Published as a conference paper at ICLR 2026
MUON with an exact polar step computed by SVD. All results are derived under the same smoothness and counded-varianee assumptions, and progress is measured by the nuclear norm of the gradient, so the rates directly comparable.
The convergence bound in Theorem 3 for SGD with momentum (Theorem 3 is, to our knowledge, new and may be of independent interest. It provides clean reference point for vector-based updates under the same stationarity metric.
Theorem (Convergence SGD with momentum). Suppose Assumptions anddn 2 hold, and run
SGD with momentum. Choosing min
1-
(1-3)a and B 11 1-min LDB
ov 1}, the follow-
41.
ing holds
T Σ
E E
(0(())))
1/4
LLD T
BT
Consequently, SGD with momentum attains e-stationarity with iteration complexity of o( (mmax
FLD
LD 2 Be }).
Note that the rate for MUON with SVD (Theorem 4 is obtained as special case of MUON with NEWTON-ScHULL analysis by setting the polar-aprroximation error zero (i.e., replac- ing NEWTON-SCIULZ with an exact polar step). This "oracle" baseline clarifies the gap that NEWTON NEWTON-SCHULZ needs to close practice and is useful for interpreting the effect of a finite number of NEWTON-SCHULE steps.
Theorem (MUON with SVD). Under Assumptions 7 aand setting Ey 0 in Theorem 7 yields
( (To"L.D
1/4
LD
BIV s
BT
T
VBT
Consequently, the idealized MUON with SVD attains e-stationarity with iteration complexity of 202LD
max
LD
:2g3
BeT
Be
Discussion of Theorems3aand Relative to SGD with momentum, the SVD-based polar variant of MUON sharpens the rank dependence in the leading ((1/72/4) term from 1/2 to 1/4 under the same nuclear-norm stationarity metric. Geometrically, the polar step aligns the update with the leading singular structure of the gradient (via spectral-nucleat -nuclear duality), converting Frobenius- aligned descent direction into one that is optimally aligned for the nuclear norm, thereby sharpening the r dependence.
Turning to the original MUON with NEWTON-SCHUIE Theorem shows that iteration com- plexity matches that of the SVD-based polar variant (Theorem 4) to multiplicative factor Xą- By Theorem 2 Xa - doubly exponentially fast the number of NEWTON-SCHUL2 steps q (and improves with the polynomial degree k). Consequently, even a small yields rates that essen- tially indistinguishable from the ideal SVD-based baseline in terms of iteration complexity. Since each NEWTON- TWNN--CHHLLZZ step uses only matrix multiplications (and avoids SVD), the per-iteration cost is much lower, which explains the superior wall-clock performance the original SVD-free MUON observed in practice. Moreover, because MUON with few NEWTON-SCHUL2 steps verges rapidly to the SVD--aaed variant, the sharper rank dependence established for the SVD-based Muon-which provides comparative advantage SGD with momentum momentum-eeffcctive effectively carries over to MUON with NEWTON-SCHULZ, while preserving its computational efficiency.
5 NUMERICAL EXPERIMENTS
Setup. We conduct numerical experiment with the CIFAR-10 (50k/10K) dataset and CNN model, specifically CIFARNET, which has approximately 2M parameters. We compare optimizers: SGD with momentum (baseline), idealized MUON with SVD, and MUON with NEWTON SWWTON-SCHULZ (q (1, 3)). For the CAATON-SCHUIT step sweep, we use the NEWTON-SCHUS with degree 2. We 50 epochs, and the batch size is B 512. Results are plotted in Fig.
polynomial p.
8

---

Published as a conference paper at ICLR 2026
SGD-M
(UDD(SVD)
Muon(q-1)
Muon(q-2)
Muan(q-3)
Train Loss vs. Epach
Test Loss vs. Epoch
1.25
0.75
1.20
0.70
1.15
0.65
1.10
0.60
Train
1.05
0.55
1.00
0.50
0.95
0.45
0.90
0.40
10
35
40
10
35
40
45
50
Epoch
Epoch
Train Loss VS. Wall- Coll-Clock Time
Thist Loss vs. Tall-Clock Time
1.25
0.75
1.20
0.70
1.15
0.65
1.10
0.60
Train
1.05
0.55
1.00
0.50
0.95
0.45
0.90
0.40
20s
30s
10s
20s
30s
40s
Woll-Clock Time (5)
Wall-Clock Time (s)
Figure 1: NEWTON-SCHULZ steps (ạ) ablation. MUON with NoWTON-SCHULZ2org (1,2,3) MUON (SVD) and SGD with momentum (SGD-M, baseline).
Performance is assessed by plotting the training loss (left column) and test loss (right column) over epochs (top row) as well as the cumulative wall-clock time (bottom row). Results represent the average of five runs with different random seeds, including standard deviations. More detailed numerical settings are described in Appendix [.I]
Ablation on NEWTON-SCHULZ step 9. As q increases, the learning dynamics per epoch steadily improve: MUON with = 1 already outperforms SGD-M, and {2.3) nearly coincides with the SVD-based MUON both train loss and test loss, in line with Theorems [ and which state that the NEWTON-SCHUL
variant matches the SVD iteration complexity up factor Xς - that decays doubly exponentially in q. At the same time, the bottom row of Fig. shows that MUON with q 2 or 3 reaches given test loss substantially faster in wall-clock time than the SVD variant, reflecting the lower per-iteration cost of the NEWTON-SCHULZ update.
We additionally performed numerical experiments on various datasets using models at different scales (with the number of parameters indicated in parentheses): multilayer perceptron (MLP) with 0.5M parameters on MNIST; ResNet-18 (11.2M) on CIFAR-100; WideResNee-288- 10 (36.6M) on Tiny- ImageNe;: NanoGPT (124M, Transformer) on Fine Web; and GPT-2-based model (1.3B, Transformer) on Web. All additional experimental results are presented in Appendix G
that varies the degree of the polynomial degree K. We perform a controlled ablation polynomial while fixing the number of NEWTON-SCHULZ steps to 3 for all variants. Increasing the degree E (1....,5) im- proves optimization (the loss drops faster at fixed epoch) but lengthens each step, yielding clear accuracy time trade-off (See Appendix H.T).. This mirrors the theory that the residual contracts as 55+1 & g (Lemma 3). while computation scales with polynomial evaluations.
Ablation on NEWTON-SCHULZ
Rank dependence. We vary the monitored layer's effective rank r E (16. 32, 64, 128, 216} and plot the avvrree [W])] 0.3 (grows with r), whereas MUON and its variants are nearly flat. These observations are precisely in line with Theorem and Theorem which state that orthogonalizing momentum reduces the rank
log-log scale. SGD-M shows a positive slope of approximately

---

Published as a conference paper at ICLR 2026
dependence of the leading O(1/T1/4) term from r!/2 to r//4, The more detailed experimental settings and results are deferred to Appendix|H. H.2
CONCLUSION
We analyzed practical MUON with finite NEWTON-SCHHE steps and proved nonconvex conver- gence to e-stationarity with iteration complexity matching the SVD-polar idealization, up factor Xy that shrinks doubly exponentially in (and improves with k). Thus, few NEWTON-SCHULE steps recover the idealized rate while remaining SVD-free and GPU--riendll. We also provided baselines for SGD with momentum and the SVD>ppolr variant, showing that MUON weakens rank dependence under the same metric ccoosing the theory-practin gap and explaining its practical performance.
USE OF LARGE LANGUAGE MODELS
Large Language Models (LLMs) are used solely as assistive tools for writing. Specifically, we employed an LLM to improve the clarity, grammar, and style exposition. No part of the research ideation, algorithm design, theoretical analysis, experimental results involved the use of LLMs. The authors take full responsibility for the content of the paper.
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea (NRF) grant and the Institute of Information & communications Technology Planning & Evaluation (UTP) grant both funded by the Korea government (MSIT) (No. RS-2022-NROT 1853, RS-2023-00222663, RS-2025- 25463302) and by AI-Bio Research Grant through Scoul National University.
REFERENCES
Kwangjun Ahn and Byron Xu Dion. A communication-effiieenn optimizer for large models. arXiv preprint arXiv:2504.05290 05295, 2025.
Kang An, Yuxing Liu, Rui Pan, Yi Ren, Shiqian Ma, Donald Goldfarb, and Tong Zhang. Asgo: Adaptive structured gradient optimization. arXiv preprint arXiv:2503.20762, 2025.
Amir Beck. First-order methods in oplimization. SIAM, 2017.
Jeremy Bernstein and Laker Newhouse, Old optimizer, new norm: An anthology. arXiv preprint arXiv:2409.203: 20325, 2024.
Léon Bottou, Frank Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223-311, 2018.
Stephen P Boyd and Lieven Vandenberghe. Conver optimization. Cambridge university press, 2004.
Emmanuel Candes and Benjamin Recht. Exact matrix completion via convex optimization. Com- munications of the ACM, 55(6):111-13 2012.
Lizhang Chen, Jonathan Li, and Qiang Liu. Muon optimizes under spectral norm constraints. arXiv preprint ar arXivv2506.15054 2025.
Saeed Ghadimi and Guanghui Lan. Stochastic first-and zeroth methods for nonconvex stochas- tic programming. SIAM journal on optimization, 23(4):2341-2366 2013.
Gene H Golub and Christian Reinsch. Singular value decomposition and least squares solutions, In Linear algebra, pp. 134-151. Springer, 1971.
Vincet Gupta, Tomer Koren, and Yoram Singer. Shampoo: Preconditioned stochastic tensor opti-
mization. In International Conference on Machine Learning, pp. 1842-1850. PMLR, 2018.
Martin Jaggi. Revisiting frank-wolfe: Projection-fie free sparse convex optimization. In International conference on machine learning, pp. 427-435. PMLR, 2013.
10

---

Published as a conference paper at ICLR 2026
Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https:
//kellerjordan, github. io postssmmunn
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Dmitry Kovalev. Understanding gradient orthogonalization for deep learning via non-euclidean trust-region optimization. arXiv preprint arXiv:2503. 12645, 2025.
Jiaxiang Li and Mingyi Hong. A note on the convergence of muon and further. arXiv e-prints, pp. arXiv-2502, 2025.
Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, et al. Muon is scalable for Ilm training. arXiv preprint
arXiv:2502.16982 2025a.
Liming Liu, Zhenghao Xu, Zixuan Zhang, Hao Kang. Zichong Li, Chen Liang, Weizhu Chen, and Tuo Zhao. Cosmos: A hybrid adaptive optimizer for memory-efficient training of llms. arXiv preprint arX1v22502.77410, 2025b.
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization, 1954) 1609, 2009.
Yurii Nesterov. Introductory lectures convex optimization: A basic course, volume 87, Springer Science & Business Media, 2013.
Thomas Pethick, Wanyun Xie, Kimon Antonakopoulos, Zhenyu Zhu, Antonio Silveti-Falls, and Volkan Cevher. Training deep learning models with norm-constrained Imos. In Forty-second International Conference on Machine Learning, 2025a.
Thomas Pethick, Wanyun Xie, Mete Erdogan, Kimon Antonakopoulos, Antonio Silveti-Falks Falls, and Volkan Cevher. Generalized gradient norm clipping & non-euclidean (1 0, s-smootthness. In 39th Conference on Neural Information Processing Systems (NeurIPS 2025), 2025b.
Sashank J Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. In International Conference on Learning Representations, 2018.
Artem Riabinin, Egor Shulgin, Kaja Gruntkowska, and Peter Richtárik. Gluon: Making muon & scion great abrinn(r theory and practice of Imo-based optimizers for 1mss). arXiv preprint arXiv:2505.13416. 2025.
Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathemati- cal statistics, pp. 400-407, 1951.
Naoki Sato, Hiroki Naganuma, and Hideaki liduka. Analysis of muon's convergence and critical batch size. arXiv preprint arXiv:2507.01598 2025.
Ishaan Shah, Anthony M Polloreno, Karl Stratos, Philip Monk, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Ashish Tanwer, Darsh Shah, et al. Practical efficiency of muon for pretraining, arXiv preprint arXiv:2505.02222, 2025.
Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszezynski. Lectures on stochastic program- ming: modeling and theory. SIAM, 2021.
Wei Shen, Ruichuan Huang, Minhui Huang, Cong Shen, and Jiawei Zhang. On the convergence analysis of muon. arXiv preprini 23X2:22505.23737. 2025.
Mark Tuddenham, Adam Prünel-Bennett, and Jonathan Hare. Orthogonalising gradients speed up neural network optimisation. arXiv preprint arXiv:2202.07052, 2022.
Amund Tveit, Bjørn Remseth, and Arve Skogvold. Muon optimizer accelerates grokking. arXiv preprint ar 2504 16041, 2025.
11

---

Published as a conference paper at ICLR 2026
Nikhil Vyas, Depen Morwani, Rosie Zhao, Mujin Kwun, Itai Shapira, David Brandfonbrener, Lucas Janson, and Sham Kakade. Soap: Improving and stabilizing shampoo using adam. arXiv preprint arXiv:2409. 11321, 2024.
Fangyu Zou, Li Shen, Zequn Jie, Weizhong Zhang, and Wei Liu. sufficient condition for conver- gences of adam and rmsprop. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 11127-11135, 2019.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 BASIC FACTS FOR MATRIX NORMS
Definition (Schatten Norms), For each p E [1, oo], the Schatten-p norm is defined as || ox where (x(X) the singular values of X.
Nuclear/trace norm: || = ||x||_ = Σ. oi() (Sum of singular values)
spectral/operator norm: |sx ||-[|lop max, (X) (largest singular value)
Frobenius norm: | = || VE,
For any conjugate pairs (p. 9). i.e., 1 + - the norms lls,, and || lls,, dual each other. In particular, the nuclear norm and the spectral norm are duals.
p g
Lemma 4 (Hölder's inequality). For any A, B Rm
B) S |||||- Blpp
Proof. Let SVD(A) = (U,
B) tr(A B) (r((UEV)) - tr(EUT BV)
Let's define new matrix C = U BV. Since U and V are orthogonal matrices, their operator norm is 1.
[CCllp -100 BV BV S |10
1- |B 1 ||
Since E is diagonal, tr(EC) = E, o,A)Cr where (A) is the i-th singular value of A.
(A, |t((C))]= Σ (A(AC Σ o.(AIICH S Eo(())(Clll ||A||-||C|lopp
Combining inequalities, we have F) |||||- | lop-
Lemma 5. For any B,B E pmxx and any constant 3 E (0, 1).
1 |
I <
+ 1 B |
B
Proof. By Young's inequality (i.e., (x, constant e > 0,
| + 1llll with 0), for any positive
B = |A 2 + B1 + + B)F
B <||A|| 1 1
< |
1+ :) B1R
(1 =11+)||AA1
If we choose
1-3 where 3 E (0, 1), we get
1
||A + B < / |
+ T T 1
Proposition 2. Let X E Rmxn with r min nt). Denote that its singular values are (o, Y Then the following holds:
(i) | Vtr(X X) = Eit X
13

---

Published as a conference paper at ICLR 2026
Proof. Let SVD(X) = (U, Then
tr(xTx) = -
=ir(V = (r(TT) = E = 1|
Since (x = Xij, we have
(xTx) = Σ (x (xT)) X3x = Σ X
1=1
1=1
Hence, tr(xTxx= E [x=((XX)) X) Σド= Σ X
(i)) P Polar(XX, then P) ||
Proof: Let SVD (U,E, Then P Polar(X) -V
P) = (UEV VVEVT.UV' == (t ( = Ur(V
=tr(VEVT) to (v VΣ) tr [x) ||||,
(iii) |X|| I /r |||||--
Proof. By Cauchy-Schwarr Schwarz incquality,
| = Σ (x( S
Σ =x(x)== -
1=1
1=1
(iv) |X| lpp S | & ||||_
Proof. Since singular values are always non-negative (o:(X) 2 0),
2
(x)x),(X))² $ Σ ox(x) <
Σ [o((x)
Taking the square root, we get || I |||F S ||||..
(v) The polar factor is scale-invariant: Polar(cX) = Polar X for all c <>> 0.
Proof. Let SVD(X) V.). Then P Polar(X) UVT Also, SVD(cX) (U,CE,V V), so Polar(cX)= = UV Thus, Polar(cX) = Polar(X) for all c> 0. 1
A.2 LEMMAS UNDER ASSUMPTIONS
A.2.1 ASSUMPTION II
Definition 5 (L-smoothness), A differentiable function f R + R is L-smooth if its gradient L-Lipschitz continuous, i.e., for all X, E mmmx
[V f(YII < -
We smoothness with respect the operator norm (and the nuclear norm its dual).
14

---

Published as a conference paper at ICLR 2026
Lemma 6 (Descent Lemma). Let f: R - R be L-smooth. Then, for all X, Y R
L I1 x x
f(V) & 1(X) + (VI(X), -X)F +
2
Proof. Define an auxiliary scalar function y [0, 1] R by considering the value of s along the line segment from X to Y:
g(s) :- f(X +(Y (X))
By the fundamental theorem of calculus,
f(Y) - f(X) 9(() ['' g'(s) ds
Using the chain rule, g6()) = (VS(X + s(Y - X)), X)p. Substituting this back into the integral equation gives:
((Y) - ((X) = (VS(X + -X), rds
Add and subtract (VS(X), xx)= So (VS(X), Y F)s we have:
- = x)F+ + /. (V s(Y - x)) V - X)
-
We now use the generalized Cauchy- -cchar- inequality or Hölder's incquality (Lemma 4 which
states that for any matrices A, B |2 and (A, B) S || the nuclear norm (l) [|*)
is the dual norm to the operator norm (1l (lop) Applying this to the integrand:
(VX(X + s(Y X)) VS(X), Y - xX) < VV[X + + - X)) V - -Xllop
By the L-smoothness of f:
IV + s(Y X)) (x))]l S +||((Y X)1lop Ls||Y X
Combining these inequalities, obtain bound the integrand, yielding the final form:
f(x) = (V / Ls| X1e = (Vf(x),) - x)) + L l - 11ep
Lemma (Gradient-gay) inequality). Under Assumption 7 for any W, we have
[Vf(W)] < 2L (f(W) f*)
Proof. In the descent lemma (Lemma 60. let = x - -VV/(XX) Then, we have
X (x- 11-1(s(x) < f(x)+ Vf(X), / v x) + I. | -
V
op
=((X) T, | + 1 | |V())
2T.
S S(X) 7 [I 21. |x)
f(X) 2L [Vx())
where the last inequality is due to Proposition
iv). Since f < 1, V.())), we have
[V)( < 2L (f(W) - f*)
15

---

Published as a conference paper at ICLR 2026
A.2.2 ASSUMPTION 2
Lemma (Unbiasedness and bounded variance). Assumption holds, then
E [[G,, ((W:)il S o² and E||G. - ([(Wolll /B
B
where B is the batch size.
Proof. Since Gt = Dim with = V.(Weist.))
E [ V =E
(f((W.:).) - -V((W))
B
B
E
(V((W: - vS(W)))
B
0²
E
vf(Wi)
B
(WW) and independence. The last inequality is due to V[IVV/(W;&,.) ((W) 0². According to Jensen's inequality, we have
where the last equality is due i.i.d. uniform sampling with W[VS(W, &t,i)
B[IGG, [/(W)lle) S BBIIG V/(W)) VB
16

---

Published as a conference paper at ICLR 2026
A.3 NEWTON-SCHULZ POLYNOMIAL
NEWTON-SCHULZ steps orthogonalize a matrix X via NEWTON-SCHULZ steps applied to A XXT. Define NEWTON-SCHULZ polynomial ps. the degree is The following are the properties of Ps along with their proofs.
NEWTON-SCHULZ polynomial px-
For degree N, the NEWTON-SCHUR polynomial is the Taylor truncation of 1/VA at 1, i.e.,
(1) = d'
A
1/2
dAs
A=1
for s = I....,K. K. The explicit form of the NEWTON-SCHULZ polynomial for degree is
= (2s)
Pa
Σ c,(1-
4'(s))2 > 0.
==0
Equivalently, with reparametrization u 1 X [0, 1] and p u) Σx= C.u
Proposition |
(Properties of Pr). For À E [0, 1]:
Positivity, (A) > and Pie (A) 2 1 with equality and only if X
Monotonicity of Let (()) Appx (A)]2: then have non-decreasing on [0, and r(1) 1.
Consequently, for any symmetric A > 0 with spectrum in [0, 1], the NEWTON-SCHULZ update A + Pre (A) satisfies Ap, 1: NEWTON-SCHULE steps preserve the unit spectral ball (see Appendix A.3).
proof.
Positivity. Pa((A) > and (A)(A) 21 with equality if and only X = 1.
Proof. Separating the first term (for from the summation that defines px (A):
K
(A) Σ c. x co A) (1-1-1) X)*
The coefficient co is (2-0))
To 1. The rest of the polynomial is the sum Ee=1 Co - A)". For any
term in this sum and for any x E [o. 11. the coefficients c, are strictly positive for all 2 1. Also, for any exponent s 2 1, (1 x) is non-negative the range [0, 1]. Each term e, - X) a product of positive number and a non-negative number, which means each term is non-negative. Hence,
(A) = 1+ Σ - -(1-))2210-1
This proves that (A) for all X [0, 1]. It is also trivially true that (A) 0.
Monotonicity of r(A) = A(px
Proof. First note T(1) = 1 (1)2 = For monotonicity, differentiate:
T(A) Px (A)2 2bpx. (A) (A( (A) + (A)..
Set u = 1 À and define q(u) := (1 (1- - u) = -E u2 Then
(a()) = qu)),
P(((A) = 4(1 - >) -d((2))
A == 1 u.
d)
Hence
7(()) q(u) (g(u) 2(1 m)g((u))
5=)(w)
17

---

Published as a conference paper at ICLR 2026
Now, compute (t)) in closed form. Since
q(u)
cal
((u) = Σ
8Cg u a-
==0
==1
we obtain
(u))
cou - 2(1 u) Σ
80g 1==²
==1
Expanding and collecting coefficients of u² gives
S(u) = (co - 2c1) +
Σ
((2t + 1)e. 2(t 1)c44) + (2x + 1)e,a".
1=1
Using the ratio identity
Ca+1
(2s + 2)!(4))
(2s 22 +1)
+ 2(s + 1)' 4(s+1) 1)2
Cs
O,..., K 1.
2/
(2t + 1)c: 2(t 1)441 (2t + 1)c+ 2(t 1)
0,
2(1 + 1)
and also co - 20, = 1 = 0. Therefore, all coefficients up to degree k 1 vanish, and we are left with
S(u) (2x + 1) 2 0 for [0,
with strict positivity for > when k 2 1. Since q()) on [0, 1J. we conclude
T'(A) = (()) 2 for all À E [0,1].
i.e., non-decreasing on [0, 1). and (A) > 0 for X (0,1) when & 2 1. Together with 7(1) 1, this proves the proposition.
NS step preserves the unit spectral ball. Let 2 0 be symmetric with spectrum o(4) C [0, 1].
By spectral calculus,
Pa (A) U diag (Ae)) UT.
where A = U ( Thus
(A) (A)llop = max (MiPa(A.))) max T (At) S r(1 1,
because is decresssn on [0, 1]. Hence, the ball into itself.
update maps the unit spectral
18

---

Published as a conference paper at ICLR 2026
B MUON WITH FINITE NEWTON-SCHULZ ITERATION
Algorithm MUON with NEWTON-SCHULL Orthogonalization
Require: learning rate 7) > O, momentum E [o. 1), NEWTON-SCHULL steps 9 polynomial Px with degree K, batch size B, total iteration T. N,
1: Initialize: MM <- 0, Wo R
2: for /= to do
4: Mr + 3-M,-1 1= ([(WW-ii
3: G, < + -
5:
Xx.0 f .M/as; with a₁ = max [
(scaling ensures || I 1)
6:
for = =1 do
7:
Xt,j + 1 X1,5-1
8:
end for
9:
Or 6 Xt.9
W, f- Wi-1 110
11: end for
Proof of Theorem 7 First, we introduce the scaled EMA momentum: N; := (1- M,, Then, we get N₂ 3N, Pe be the polar factor of M,, i.., P. = Polar((M) (M). Hence, P. == Polar(N,) = Polar (M,).
G)G, Note that the polar factor is scale-invariant (Proposition 2 v)). Let
By Assumption
we start from descent lemma (Lemma 6 Since W. = W n10,, we have [
J(W.) S(W.-) + (VS(W-)). W1-1)F I|W Wi-1llap
=((W-1)) (Vf((W 0,) "" 10 10
=S(Wi-1) N/ 01) (/N V/(W.,), 0,) 2 " 110r
S(W--1 ( N. OFF - N VS(W: 11). + 0 "
L
-[WW-1) - (N. p NN. P: 1/ olop + 2" 110:1000
L
=((W-) N. + NN. P. 0.) ~ N V + クタ1 n
2
- S(WL-)) N.I - + > - O. + N/ - v/(W-i))ll))])),>> + 1 110
where the second incquality and the third incquality are due to Hölder's inequality (Lemma equality is due to (N), Pt) = [|N.].. (Proposition 2
4 and the last
Now, define the polar approximation error, which is the enror between the exact polar factor P. Polar(N:) Polar(M:) and the actual step Ot generated by q-step NEWTON-SCHULZ steps, i.e., O: NEWTON SCHULZ(Mc,,)) measured in the operator norm.
Eig 1 Pallop and Eq sup
Since ||Prllop 5 1, 11eellopppppppp 5 + Et. < +1++60 Eg. we have
S(Wa) s f(W(-1) 1111 ||: Oz [ep)) + N/I.N V/(W:-))l-1 + 1 01
V L 2 Et.))
< <(WW-1 7/1 -., -
S <((W-1) - - - - (/((1) -NIl-
+1(11 2 11 V 11
V (((W-))) L, 1 st,y²
L
= =(WW-1) (1 - st.,) |V/
2
where the last inequality is due to - |/|| < B|| + |4- - Bl| and the last equality holds because (1
Et.a) + (1 + Ex.4) 2. Since εt.o < cq. we arrive at the clean one-step inequality
f(W)) f(W-1) - -
N.ll. 4 =g)
(1)
19

---

Published as a conference paper at ICLR 2026
Rearranging and laking the expectation yields
E| V -
2 V - N.ll- Ln(1 + tg I- Eg
(2)
v(1 Ex))
2(1 - Eg)
Bounding - -Nll--- -Nell--
order to bound [[IV)((W--)) N. we introduce the true scaled momentum N, defined by the true
(full-batch) gradient V V/(W)) instead of G. each step
= - /)V for > and No = O.
Note that N₂ .3N + (1 B8G, for and No O.
Then we can decompose F N as
/ - - ((W - Nell-
(3)
Tomm (A)
Tern (B)
Let Bounding the Term (A). Let D, = -:= - N.ll-
From the recursion,
vf(Wi-1) No B(V BWW N.-1)
Hence, we have
D. 8 (VVf( - -M-1ll-
- V [W(
- VS(W,-))|l
3D. + /[V V/(Wi-)))
Applying Assumption | and using the fact that |0 pp - 11+ 1. Σ 1 E.g we have
[V/((--1) < W Wi On 1|op Lny( Eq)
Hence, we have the following recursion.
D. BD BLm +,
Since No 0, we have
D1 [VV)())) N1 ||- = ///(W)) - (BN, + (1 --3VV./(WW))l = /(])]l
By unrolling the recursion and taking the expectation, get
B[D.] < 8 ((Wolll( + Σ stmimi ('))l((W)]]l + 3Li((1 + 8q)
(4)
1-8
Bounding the Term (B). When unroll the EMA recursion of both N₂ and Nt,
N; = 3N + B). No + (1- No
8) Σ B
N; = 3N + (1 (V.))) 3
VVWW)
Then, compute the expectation of the Frobenius bias caused by Mr and Me,
B[[|N - NeIl= 3)E s' vS(W.))
(S)
Applying Jensen's 's inequality and the linearity of expectation gives
Σ (G (G. v)(W3))
(1 - 8-BBB
5
(1 3)2B
gr (G: vf(W:))
(1 B)
E
(2(--1) |C. vS(W)))]13
(1 - B)
Σ
8a(1- v
20

---

Published as a conference paper at ICLR 2026
By Lemma 8 Eq. becomes
EIJ N.lle) (1 (11-)) () 1
Σ gon BIIG, - f/W.)
1- B2e 0²
1 3 1+3
5 2)
1- 32 B
Using the fact that ||||| S X for X E R'Xx" with r min(m,n) n3 (Proposition 2 ii)), we have
1-8 3 Vro V 1 3 /B
B[[ Nell-] S - Nolle]
(6)
Plugging Eq. and (q.6) into Eq. 3), we obtain
(La(1+ ( + V
1 1 B Vra 11+ B /B
V[(W-))))))> N
(7)
1 1-B
Averaging and tuning. Plugging Eq. into (q. we get
[ [[[ f(W))
n(1 Eq)
BLa(11 + Eq)
1-8 Via 1- + + + V 11+ 3 vB
La + sq)² ) 2(1 Eq)
Eq
Averaging over t = 1.... T, we obtain
D
2
3Lm(11 + +eg)
1 - B Vro
B [W]
+
Tny(1 - Eq)
1
- Eq
1- 3
1 + B /B
2
1
T
Ln( E²)²
3'
+
1
T
2(1 E.g
Using Proposition [ii) and Lemma 7 /f(W))ll- S VV S V2rLD, where f(Wo)- Then, we have
Σ [Wl
D
3Lm(1 + e.) 2
B-B vio I+ B VB
Tnj(1 Eq)
Eg
1- B
23 2
Ln + E.y) )
(1- TT-Eg)((11 B)
2(1 Eq)
where = f(Wo) - fr By setting 1
(1-3)D we obtain T1.
r
T
D
La (1 11a))
28Lm(1 +e,)
20
1 1- 3
28V2rLD
+
Eq
TTj
2
1-B
/B
I+ 3
T(1 3)
LD
1 2/(1 + Eg)
(1+
20 20 VB
1 B +
23
+
1 Eq
T
VT-B
2
1 B
T(1 B)
Setting
/ = 1 min
VLDE oVFT 1
we get
T
E ((We-)))l
T t
(1+ Eq) 1 3
LD 1+ 23(1
LD
20
1 1- 3
23
< - Eq
2
T
VI-3
V +
vB V 1+3
+
T(1 - 3)
1 1-E Eq
(1+ ) -
V T
LD
+
1 1311 +zg+ +
VT+B
2
(To2LD) 1/4
BT
+
2223 or
VBT
0 -0
LD
or + VB
LDD 1/4
T +
(8)
1-E
BT
21

---

Published as a conference paper at ICLR 2026
From NEWTON-SCHULZ residual to factor Xę- By the NEWTON-SCHULZ residual-error link (Lemmal and the residual contraction (Lemma)) as assembled in Corollary we have
Eq sup Ei.g sup(1 vI d.,e) - 1
60 1
Hence, we bound the factor as
Xo (1 (4) 1 d0 1 1-1/2
with do = sup, 66,0 < 1 (Remark D Finally, we conclude from Eq. (8) that
T (IW X. 0..0 OVT
T
LD T
( 1/4)
1
VBT
+
BT
Xy
1
with the constant factor bounded
X4
1
[1- 60 xx+1)
1-1/2 1/2
Finally, can find e-nuclear norm stationary point of with complexity of
LD
Xxr² 0²
r
})
max
6²
Be²
Be
22

---

Published as a conference paper at ICLR 2026
C MUON WITH SVD AND SGD WITH MOMENTUM
C.1 THEOREM (MUON WITH SVD)
Algorithm MUON with SVD
Require: learning rate > O, momentum 3 E [0, 1), NEWTON-SCHULZ steps E N, batch size
B. Total iteration T'.
1: Initialize: Mo < 0, Wo R
2: for 1 to T do
3: G₁ 4- 13 - (.(WW-ritt.)))
4: Mr f- 13M,-1 1 +G+
5:
O₂ < (Polar(M)
(SVD(M)) (Ut, Ex, Vi), then Polar(M) U.V))
6:
W, f Wi-1 - n0
7: end for
8: return Wr.
Proof of Theorem First, we introduce the scaled EMA momentum: N; := (1 3)M, Then, we
get N, 3N, (1 3)G,, Note that the polar factor is scale-invariant (Proposition 2 v)). Let Pt be the polar factor of M,, i.e., P. = Polar( M.). Hence, P: = Polar(N,) = Polar (M.)
By Assumption 1
start from descent lemma (Lemma 6) Since W. W nO we have
L [ W1
S(W,) < 1(W,11) + (VS(W.-),, W, - W(-1 +
2
T.
= =((W-1) (yl (V P + "''2
2
-((W--)- 7 (V P + 2 P
where the last equality due to O, = P. = Polar(M:). Let SVD( = (U,, Ee, V/).. Then P: Polar (M,) U.V. Since P. (olar(M)) is partial isometry, |Prllop I 1 (and |Pllop 1 if
rank(M) > 0). Then, we have
L f(W) S f(W-11) (VF((W-)))))) 2
Ln²
= =(WW-11 - T/,, 7/N, V.(WW-1), Pt) +
2
Ln?
= =((W--1) - N/l. + (N, Pl) +
where the last equality is due to (Ne, Pt)r = || (Proposition ii)). By Hölder's inequality (Lemma 4 and the triangle inequality, we have
f(W,) S f(W.-1) w/l N.II + Vl (WWW Lnp
Lnp
SS(W-1) - // /// +>/l Nell. + nnllN V/(We-1)ll+ +
=((W,1) W/ + 2// - Noll- + Ln?
2
Rearranging and taking expectation gives
E| ((W:-))\1 B[f(W,-1) f(WW))
Ln
+ 2E VF - NoII+
(9)
2
Bounding ((W,))) N.II--
In order to bound V. N.ll... we introduce the true scaled momentum N, defined by the true (full-batch) gradient V/(W,) instead of G, for each step t:
23

---

Published as a conference paper at ICLR 2026
N₁ -M8-1 - (1- for > 0 and No 0.
Note that N, 3NN-1 (1 -3)G, for t > 0 and No 0.
Then we can decompose V Nell+] as
E B[[V)(( N.l1-. ((W + -elll
(10)
(A)
(B)
Bounding the Term (A). Let D. = [IVf(W No From the recursion,
Vf(Wi-1) - -N-))
Hence, we have
D: = BlV - -N-ll
- ( -N-1-ll+++ (
+ (-W
Applying Assumption D and using the fact that 00- PPPt-lllop S 1, we have
\V - (2WW-))).. - L Wi-1 Wr-2llop Ln)[O 1lap S Ln
Hence, we have the following recursion,
D. S BD BTn
Since No 0, we have
D1 = [(W)) - Nill+ = [[V)(W)) (9No + (1- = = (lW))ll
By unrolling the recursion and taking the expectation, we get
E[D,] 8 ((W)lll + Σ L (VVS(W)))
BLn
(11)
Bounding the Term (B). When we unroll the EMA recursion of both N, and N.
N₂
+ 3)
8 g
N, = BN,1 + +1 (1- No (1 3)
g (WW,)
Then, we compute the expectation of the Frobenius norm bias caused by M, and M,
[| - N.1l=] s (1 B)
Σ B (G, - VV(()
(12)
Applying Jensen's inequality and the linearity of expectation gives
2
Σ 3 IG. v((W))
< (1 - 3)2E
Σ 3r (G; vf(W))
= (1 -
(1-3), 3)
32(1-1) |[ - vS(W)))
(1 -1- 3)
Σ
32 320-- v)(W))
24

---

Published as a conference paper at ICLR 2026
By Lemma 8 Eq 12 becomes
Bi Noll=] S 1 /
3alt gan- V (WW))
B2t 0²
1
$ { 1 1 - 32 B / 1+ 3 VB
-
Using the fact that |||||+ & /r | for X E Rmxx with r= minfmm nn (Proposition fii),, we have
1- Vro
B||lN, Noll=] S VEBIIN, Nollel 11+B 3 vB
(13)
Plugging Eq.
11
and Eq. 13 into Eq.
10
we obtain
BLm
1 B vio
[ N.1l.]
(14)
1-B
V
1+ VB
Averaging and tuning. Plugging Eq. 14) into Eq.
we get
(WW B[f(W,-1)
f(W))
BLn 1-3
/ro
Ln
+2
+
+
1 VB B
Averaging over 1,... T, we obtain
1
D
BLn
1 1 Vio
2 T T
Ln
(I) - Tn
+2
+
B" | /f +
T
1
3
1+ 3 /B
2
1=1
Using Proposition 2 iii and Lemma f(Wo) S" Then, have
[V S vFLIVS(Wo)ll 3 VrrLD, where D =
T
D TT
BLn
1 B Vio
2B/2rLD
Ln
+2
+
+
+
T
1-B
I+ VB
T(1
2
1=1
where D = f(Wo) - f By setting 1)
/
(1-3)D TT
we obtain
D
Ln
2BL
20 Vr
1 + 23/2rLD
Elll (W
+
Tm
2
1
VB
1 3
T(1-3)
LD
1
V1-B B
V1
20 2
1 B
28v2rLD
V T
VI- B
2
VB
1 B
T(1 3)
Setting 8 = min
VLDBB oVTT
we get
T
VI-3
LD
1 23
LD
20 20
3
23
T
2
T
T
VB
1 3
T(1 111-3 -3)
VI-
3
LD
1+
+
2
ro2
1/4
+2/28
or
2
T
VT+
BT
VBT
o
LD
or
(DD'LD
1/4
(15)
T
VBT
BT
Thus, we can find an e-nuclear norm stationary point of f with complexity of
LD 1022 ro²LD)
Be })
0
max
2
B
25

---

Published as a conference paper at ICLR 2026
Algorithm 3 SGD with momentum (SGD-M)
Require: learning rate 7j > O, momentum $ E [0, 1), batch size B
1: Initialize: Mo -- 0, Wo m
2: for = to T do
3: G₁ < Σ 1= V ((W
4: M2 f- 3M G2
5: W, < Wi-1 M
6: end for
7: return Wr
C.2 THEOREM 3 (SGD WITH MOMENTUM)
Proof of Theorem
First, we introduce the scaled EMA momentum: N₂ := (1 M) Then,
we get N₂ (N + (1 B3GG with No 0 and the scaled learning rate 9
1-3 yielding
W, W N.
By Assumption 1
we start from the descent lemma (Lemma 60. Since W. WW-1 Nj, we have
L
S(W:) & ((W-1) + (VS(W.1), W, - W2-1 +
[|; Wi-1llap
2
4 7 1 N
= -
By using (a,b) 1/2 (lal1 ||6 11a b112), we obtain
7
1
L 1² Nellop
S(W)) S 1(W-11)
V
11 133
I
SS(W-
2
2
" I
L
$ f(W-11)
I 1|:
(f(Wi-1)l1
+
2
2
77(1 I.ii)
=(WW-1)
2 (
IIN, Tf(WW-))
|
< S(W,-1)
I,, V V(WW..)
where the second inequality is due || < |N F (Proposition ivv) The last inequality holds by choosing ñ S 1/L so that we can drop the last term.
Rearranging and taking expectation gives
2W( - f(W)))
E [VV K
EII (
Averaging over = to T,, we obtain
T Σ
1
2D
E (VW
E| V/(W
(16)
T 1=1
Tíj
t=1
where D = f(Wo) ft. Denote
1
2D Tī
SA
E V
SB.
SB
E - - VFWW---i (17)
T
Bounding
1 EL, EI vf(Wi-))111
We introduce the true scaled momentum N, defined by the true (full-batch) gradient (.(,) instead of G, for each step t:
N₂ BN,1 + (W,) for / > 0 and No
26

---

Published as a conference paper at ICLR 2026
Note that N, BNN-1 + (1 3)G,, for > and No- 0.
Then, we can decompose 1. - -=| E V - using l|a + bl3 S 2|1a|1 + +1111/3-
T
B(IVs(W M S
El E[] N. +2 Σ
E I Nell.)
Term (A)
Term (B)
(18)
Bounding the Term Let, == V - NN. Using the recursion for N₁, we have
Et V -
+ ( (L-))V(()))))
(VS(W,-1) - (1- -V((W))
Bei-1 + 3 (V VV +
V)((W)
Apply the variation of Young's inequality (Lemma with
3 1
+ ||e/ = || + 3 -0 V/(W.)) 1
= -1++))
(V (F(W)) 1P:
Since 1 +c= 1/2 and 1 1 1/c = 1/(1 - B). and | yl =
+ 2||x||22|[111 we get
2,32
||= < Blle, T B [V((W-)) 2 + 2(1 - B) 3) SSW.).
By Assumption
1 and the norm monotonicity -lop S 11-1 S || ||. (Proposition Riv)), we have
[V)(W)) - < (V)) - vf(W-1)) < Li| 1i-1llap
1' No & |1.
and
-VVV s L - 2 = I, I S 1
Hence, we have the following recursion,
|lellp s Blle l + 2,3
+ 2(1 - 3 ||N 11:
1-3
Taking expectation and averaging over = 1 to T, we obtain
[[IBS(W--))- N.I
(VS - N. 1|
L2
T
+
1-B 3 T Σ E 2(( 12
Elll N.
Since 1
T T ΣT V[[ - No 11:11 S co + B - N and
$ ト N 13+8
< $ - 1E| N.
(WI-))(W-.)) - N1.1
8E
+
2
3
+ 1 1-8)
8)
10 1..1. (
(+Σ)
T
T(1 - 3)
1- 3
1 3
(19)
27

---

Published as a conference paper at ICLR 2026
Bounding the Term (B). When we unroll the EMA recursion of both N₁ and No,
N₂ = 9N, -), = (1 + (1-3) 3)
Σ 3 G
i=1
N₂ - 3N,1 + (1-3)V((W,) No + (1 3)
Σ 3
S(W,))
==1
Then, we compute the expectation of the Frobenius squared norm bias caused by N, and No,
E1 - N.Il.] < E
3' No) + (1 3) (G: VS(W)))
2
Σ 8 (G; VS(W))
BII No.1 (1 - 3)²E
B No- Noll (1 3)²
Σ g2 g2(t- B ((WW)
where equalities are due to Assumption
2 which states E|G, = f(W,) so that the cross-term vanishes. Note that Bl - No.1 = O. By Lemma
8 we get
E| No [|:]] S (1- B)²
B2(1 E
((WW))
82(1-11 0²
8- 1+ 3 1+3
1- 02
S (1-
Σ
B
By averaging over / = 1...., T, we have
T
T
3 0²
- 1- 3 02 1 B B
B - Nill
S
(20)
T
1 3 B
t=1
Plugging Eq. 19 and Eq.
20 to Eq. 18
we obtain
V - Nell
T
00²
3Tlleo 13+
2T / B²
(+
| T
+2
+1 +1-8)
1+ 8 B
T(1 3)
3 11-
1 3 0²)
2,3 lo.
2
I+ 3 B
T(1 3)
87
182
+1 +1- 9)
(
Σ
T
E / (WW + 7 Σ BlI N (V
+ 1-3
3
11- 3
(21)
where the last inequality is due ||a + bb < 2||a|1 + 21b1 Eq. can be expressed as
1 1 0z
e0
(
3²
(1-3 +1 +1 3) (Sn + Sn)
Sn 2
1 3
B T(I 3)
1-8 3
3
By Lemma and Proposition Therefore,
2 iv)), we have Elleo 1b:- E| - ( < 2LD.
1 - 30²
43LD
8I.272
32
SB 2
+
+
+1 1 1)) (SA + 5
1 B B
T(1
1-3
3
1-
3
28

---

Published as a conference paper at ICLR 2026
42² 8 Let == I-3
+ +1- 8)
K and := 3 + (1- (1-B)). Then, we have -1-337
1-B
Sn S 20(SA + SB) + 43LD
(21-8 0²
(1-80)
+
(1-
1+
Hence
-1- LB S 20SA + 43LD
1 1--30²
+2
(22)
(1- 8)T
1+ 3 B
Insert Eq. (2 into Eq 17
we obtain
2D SA S Therefore, provided 0 < 1
2D
20
43LD
11- 3 0²
1 B -z ))
+
SB S
SA +
+2
Tū
Tij
1
20
1- 20
(1- B)T
2(1 20)
D
1
48LD
1- B 0² 1 B
SA 1
+2
(23)
iiT
1 40
(1 ByT
Finally, we have
2(1 - 20) (1 (1- B)D
(1-
1
4BLDD
1 B 0²
)
( <
+ 1 40
+2
1 48
1/T
(1- 3)T
1 B
(24)
Tuning 1) and B. We need 7j S 1/L L and 0 <
1. Since K = =B² + (1- 3)² E [1/2, 1]. a convenient
sufficient choice
{
11- 3 (1-8)² B)2
7 S min
ALVK
Applying Jensen's inequality and using the fact that min(m, n} (Proposition fii),, we get
|| S VrlxX|1 r for X E R with r
T-1
[] VF
T Σ
SVT
E|| f
T
T
2(1 20)
(1- (1-B)D
4,8DD
1 - B 02
S V
+
+2
1 - 40
n/T
1
40
(1- ByT
1+ B
2r 20)
(1 B)D
r
48LD
r
2(1 B) 02
+
40
T/
40
(1-3
1 40 1 3 B
where the last inequality is due va + Va + vb. By setting n min
1-8
I. "LLK f. which
(1-8)2 \
guarantees îj S 1/L and f. we obtain,
T
(1 roo ( 3)
LLD T
rBLD (1 (1-3)T 3)T
E (WW
+0
+ O
(1+ 3)B
Deterministic Term
Stochastic Torm
By setting 8 = min
VLDB o/T
we get
LLD
r3LD
rro2 B) +0 (V
T
T
+
11
(1-ByT
(1+ (11 3)B
1/4
TLD
0
T
BT
Thus, can find e-nuclear norm stationary point of I with complexity of
rLD
o max
B }
2
29

---

Published as a conference paper at ICLR 2026
D NEWTON-SCHULZ LEMMAS: PROOFS
Remark 1 (Initial residual below one).
With X M1/a, and a max(1, [||ll]) we have 60,0 [0, 1) for every moreover o.u = 0 when M, = O.
Proof. Let M, = UEVT (rank rz). Then X1, U(E2/a/)U"" so on range(M,) the eigen-
values are oī/aa E (0, 1]. If = O, set 8G,0 0. Otherwise, the minimal positive eigenvalue X - minigg, /f/oa > O, hence by Lemma 80,0 = Aii <1.
Lemma 9. factor invariance under NEWTON-SCHULAN
As NEWTON-SCHULL iterates by the polynomial Pk. i.e. X1,0 Mi/ar X1j+1 Pa ( the polar factor is invariant: Polar(M) = Polar(Xe.j) for all 2 0.
First, the polar factor is invariant under scalar multiplication. Let Let LV) (U, Then Polar(X) UVT Now, let Y cX for 0. Then SVD(Y) SVD(cX) (U, (U, V). Thus, Polar(Y) = UVT Hence, Pt = Polar(M,) = Polar(M,/on Polar(Xr (Xe,o) holds.
Proof.
Now, prove by induction on 2 0, assume that P. Polar(Xr,s). Note that one step of NEWTON-SCHULZ for polynomial is defined as
X1s+1 x(
Let SVD(X.,) = (U,E, with E > 0 (on range( Xr.i)). so that Polar(X..)) = UV Then p(UEEU JUEV =UD(E?)EV"
p(X,, X, X PUEVV (UEVT) JUEV (UEVV))JUVV Thus, p(X, X, Xt.j has the same left/right singular vectors U, V as Xt.j Hence,
Polar(X15+1) Polar(p(X.., X.,) = Polar(Xxs) = UVT
By induction, we have
P. = Polar(M,) = Polar(X.... Polar((.,,)) _... -Polar(X..)
which implies the polar factor invariance under NEWTON-SCHUI1Z Lemma 10. (Support invariance under NEWTON-SCHULZ).
As HEWTON-SCHULLE iterates by the polynomial Pa: i.e. X.,0 Mi/av. Xt,+1 Pa (Xt ) the support is invariant: range(Ms) range(X(.s) for all 2 0, and
X (y) positive definite on range(X..j).
Proof. On range(Xi,) the spectrum of A X lies in (0, 1]. PN is strictly positive on (0, 1] (Proposition D.. so pw (A) range(X(,,) is invertible. Hence
rango(X1,54) = ran = range(X (Xt,),
and by induction range(X..,) = range(M,) for all j.
Remark (Support-aware details). As in the main text, all NEWTON-SCHULA quantities are re- stricted to range( M,) (no standing full-rank assumption). Lemma 10 preserves the column space; Pt, X1,5 XT,, and II, vanish on r
Lemma I (rrthogonality residual vs. Polar approximation error).
Let X the smallest positive eigenvalue of X X.T restricted range( MM) (set X rank(M,) = 0). Then
di,g 11-A - Anin
Ergg =
V A
1 - /1 ge.g
30

---

Published as a conference paper at ICLR 2026
Proof. Let Xt.g UEVT and II, UUT with Σ = diag(o.). Then,
I1, X 1.9 U =
on range(MM) and vanishes range( - Thus, the orthogonality residual òt,y is computed 0,,0 IT, X Xi,lllo max - of X+
(25)
where t is the smallest positive eigenvalue of X X sange(M.) By the polar factor invariance under NEWTON-SCHULZ updates (Lemma we have
Er.q = [ Polar(M.)llop | UV 1lop | Illop
-imiin 0;
(26)
Combining (5. and Eq.26 we conclude
Ft.g = A = 1 - v ō6,
Lemma (Residual update).
For NEWTON- SCNTOO-SCCHLZZZ polynomial Ps. the orthogonality residual ōt, is updated by NEWTON-SCHULI per step
dejt1 <(dr.j):
u) u) (Pa(1 u)) where ((u) = - -1-
Proof. Recall that NEWTON-SCHUI-2 update defined as
X = p(X,, Xe.s j == 0, 9
9-1
where |ooooolop S Let Ars be X. Then Ar, is symmetric, because
AT,, = (X)) -xT)) = XexX Ari
Applying NEWTON NEWTON-SCHULZ update, we get
Ar,+1 =Xe,t1X = (o(AA,)
=p(Arr)Xxs X.T,)) (Ars)) = pl (A.,) Arsp(Ar..)
Since At, is symmetrie, any polynomial p(Ar,)) is also symmetric. Thus, p(A(.) p(Ar,)). Therefore, we have
AIj+1 = p(A,,)
(27)
Since Ar, symmetric, Ar, is orthogonally diagonalizable. Let (Ao, ... be the eigenvalues of Auj- Hence, we can write its spectral decomposition Ar, UAUT U diag(A)) By the key property of matrix polynomials, p(Ar.)) U U diag(r()i)U Now, substitute
the spectral decompositions of Ar, and P(Ar,)) into Eq. 27
Atj+1 (Up(A)U (UAU (UP(AA) (P(A)AA(AA)ITT
Let the new diagonal matrix be A' P(A)Ap(A). Then, the diagonal entries of A' are {2 The expression for Arj++ T which is the spectral decomposition of Ar This form explicitly shows that the eigenvalues of At,t+ are the diagonal entries of A', which are A A=
We conclude that A ((Ar))),,sp( = U diag(A)(p(A))))) U on range(M) and both II, and Ar, vanish on range(M,)
Claim: || lop & for all 2
31

---

Published as a conference paper at ICLR 2026
For O, it is trivial. Assume that ||Xt,y llop 1 holds. Then, ||ee || Il@p L. Hence, the largest singular value of Ar, in [0, 1J, which implies all eigenvalues of symmetric Ar, are in [0, 1], i.e., max ò Amax 1.
||
[ h = X = ||Atttallop = max (Aelp(Ae)]3)
Since T(A) {[((A]² is non-decreasing on [0, 1] with r(1) S (Proposition I we have
[| ll = max (T(Ar) = T (max Ar) = r(Amax) S T(1) 2
By induction, we get [|Xiijolop I 1 for all j 2 0. The claim also implies | llop S 1 for all j 2 0. Now, recall that the orthogonality residual step j is defined
o,g -1 - X = ||l,, Ac.llop
Let min
be the minimum eigenvalue of At.j. i.e, miny A1 = A I 1. Then, by the claim, we have
dej ITT Ar, lop = max Aul = X
(28)
Now, we consider the next step residual 8r,t+1,
61J+1 = |1 Ar.+tllop = max Aelp(Ae)] = max (1 Aelp(Ae)1
where the last equality is due to |Aeettllop I 1. Since T(A) {[p(A)] is non-decreasing 1] with r11) S 1, we have
btj+1= = X (p( minn) )1²
(1 - )e.r (p())(((1 81.;)]²
28 where the last equality is due to Eq. updated by NEWTON-SCHULZ as
Therefore, we conclude that the orthogonality residual is
6-5-1 ((61.)
where (bu) =1 (1 p(1 u)]
Lemma (Residual Decay by NEWTON-SCHUL..
P Polynomial). For NEWTON-SCHULA, polyno-
mial Px. (u) S ux+1 on [0, 1] where ø is function defined Lemma 2 Hence, for every and all j 2 0,
8,,41 S g++1
8z,g S 6,.0
(K+1)"
Proof. The NEWTON-SCHUj polynomial p()) of degree k is defined the truncation of the Taylor series of g(A) --1 expanded around x = 1. This means that the first k derivatives of p(A) and g(A) are identical at À 1:
n (1)
for 0, 0,1,..,.
Let's consider the variable substitution = - u. The function becomes f(u) = g(l - u) =
(1 - 11/2, The polynomial p(1 u) is then the Taylor expansion of f(u) around = 0 up to the
term u":
(()
p(1 - u)= - Σ
Σ cgu"
(29)
The derivatives of f(u) (1 112 at 0 (() (2s)
24 that the coefficients
(2s) 44 Moreover, since p(1 u) and fu) share the same first derivatives at u 0, we get
(()) (- 1) (1), for I....,
(30)
32

---

Published as a conference paper at ICLR 2026
Consider the function r(A) == 2
Claim: (A) is non-decreasing on [o, 1] with r(1) - 1.
T(1) = 1- [p())) [o(1)]² = 1 holds. Now, the derivative of T(A) is
((()) [x())]² + 2Ap(A)/(AA p(A) (p(A) + 2Xp' (2))
With = 1- u, we have y²(A) = =pp(1 - u). Then, we obtain
7()) = p(1 - u) (p(1 u) - u)p (1 u))
(31)
:=S(w)
(2s) Since all coefficients of p(1 u) are positive, i.e., Cs (xt)
0, we have
p(1 u) > for E [0, 1]
Now, it suffices prove S(u) 2 0 for E [0, 1]. From Eq. 29). compute S(u):
S(u) p(1 - u) - 2(1 u)p (1 -- u) 8
Σ cu - 2(1 u SC221 a-1
==0
s=1
(co 201) + Σ (22 + 1)c, 2(s + 1)cs+1) u² + (2x + 1)cxe"
(32)
Note that the ratio the coefficients is,
(2++22) [[++ 11122)1)+++1
Ca+1
2s 2(s +
(2a)
(1)
Therefore,
2s + 2(s + 1)
(2s + 1)c, - 2(s + 1)c,+1 (2s 1)c, 2(s +1)
C.
0,
and also co 201- = 1- 1
2. 2 0. Hence, all coefficients up to degree k cancel Eq. 32
leaving the exact factorization
(2k)
S(u) = (2x + 1)cru"
Cx 4th (x:))²
> 0.
(33)
Putting Eq. 33 into Eq. (1I. which is T'(A) = p(1 - (()) gives
7(() p(1 (2x + 1)cau",
with 1 x E [0. 1].
Since p(1 u) > 0, cx. > 0, and u² 0 on [0, 1]. we have r'(A) > for A E [0, 1]. Moreover, T'()) > 0 for [0, (because then u > 0), and r'(1) = Therefore, 7 non-decreasing on [0, 1] (in fact, it is strictly increasing on [0, 1)), claimed.
Now, we can apply Lemma 2 since TA)) non-decreasing on j0, 1] with r(1) S From Lemma know the orthogonality residual is updated according the rule:
8 = ((8r.)) =
(34)
where the function ((u) defined as:
(bu) = (1- u)(p(1 u)]
(35)
Claim: (0) = 0 for all 1,...,R. k. The function (uu) can be rewritten using f(u) = (1 - -)- and p(1 - u):
Φ(u) 1
f(u)2 1o(1 u))
(36)
33

---

Published as a conference paper at ICLR 2026
Let's define remainder function (u), which represents the difference between f(u) and its Taylor approximation p(1-u)
(u, f(u) p(1 u)
(37)
Since p(1 - u) matches the derivatives of f(u) to order k at u O, which implies Eq. (30), the remainder function and its first &. derivatives are all zero
0:
R (0) (() (1) (-1)* = 0, for ,,
(38)
Substituting p(1 - u) = f(u) - (u) from Eq. 37 to Eq.
o(u) [f()) RA, (u)
f(u)²
f(u)² -2f(u) + (u)²
fuu)
(1-
2R(u)
+ Rx(
f(u)
f(u)²
2R (u)
Ru)
(2f(u) -Rn())
Ra(w))
Ru(u).
f(u)
f(u)²
f(u)2
Let's define the term in the brackets as new function, H (u):
H(u) 2f(u) (2)
f(u)²
So, we have a simple product form: ((u) = (u) Note that (u) is well-behaved around 0 since f(0) = and (0) O. making the denominator non-zero. To find the derivatives of db (u), we use the Leibniz rule (the generalized product rule) for the s-th derivative of a product:
d (1)) du
Ra (u)
Σ
(s) R ( (()R(
Now, we evaluate this s-th derivative at u 0:
2 Σ
R (0) (
5=0
Let's consider any integer the range 0 S S k. the summation above, the index ý runs from 0 to For every term in this sum, the condition j < к holds. From Eq.(8)), know that (0) for j < Therefore, every single term in the summation contains factor of R(3) (0) which is equal to zero.
(l0)(0) (0) Σ
()))) (0) (0)
8 H (1
1=0
This result holds for all s = 1, I,..., Thus, we have proven that the first derivatives of ddu) at are all zero:
x(0) = o'(u) = .... ((() 0
This implies that the Taylor series for Φ(u) starts with term of order at #+
((x+1((0)
Φ(u)= (N + 1)! x+ + (
Find the leading term of ((u).
Recall that we can express p(1 [1 - u) using the Taylor remainder theorem:
p(1 u) f(u) Ru)
(39)
34

---

Published as a conference paper at ICLR 2026
where (u) is the remainder term, which is of order O(u*t)). Substituting Eq. (39 into the expression for o(u) in (35p:
((u)= 1- u)](((() [u)]
= 1 (1 uu](fu))² - R (u)
+ (R)(
1 2Rx(4)
1 (1 u)
2
1-u v1 -1-u 4
= 1 - [1 - 2v uux(u)) + - Rn)
2v1 - (2) (1- u) (u)
The remainder (u) is given by Ru)(u) = (1)+))(( (c) un1 for some E (0, As u + 0, the
[x+1]!
leading term of Φ(u) is determined by 2/1 - u f(++) (0) nx* The derivatives of f(u) = (1 1)-1/2 at u 0 are (0)
(x+I))
=! Thus, the leading coefficient of the Taylor series for d(u) is:
(2=)]
C₂ 2f 2 (n + 1)! (x + 1)! (x + 1)14
2 (2(k + 1))!
2
2x + 2
4
K + 1
For any и. 2 1, this coefficient is less than 1. For example, for 1, C1 In general, using Stirling's 's approximation (2r) <
l8 (t)
< 1.
16
1"
we have:
Van'
2
2k 7xx+
2 4
2
C₂ 4u+1
VT(K+1) <1
for 2 1.
k6 ax+1/
4e+1
VT(++1)
Since d(u) = Cxu*t1 + (2 and the leading coefficient C₂ is less than 1, there exists a sufficiently small Pro E (0, 1) such that <(u) I u*+1 for all u [0, px] Specifically, you can choose any E (Cx,1) 1) and take Px so that (Ø(u) Czu+t1 (0 C. C2)++1 on 10, px). Then
(uu)= S ++ S n+1
(40)
which makes the "exists px completely explicit.
Combining the recursion Eq.
34 and Eq 40) have
51, (66,,) S (
After @ steps, we obtain the final result by repeated application:
81,g S 1 < ((6,.-2))*1 2 S.. ... (x+)))
Finally, we conclude:
81gg S 000
1)-
This establishes that the orthogonality residual decays with an order of x + 1 at each step of the NEWTON-SCHULZ if NEWTON-SCHUIT polynomial is of degree K.
Corollary (Final constant factor bound). Ler du = sup, 60,0 < 1 (Remark Tj. Then for all t and
q 2 1,
ōe,g S 0
(x+1)
Eg S
1 00
((x+1)
Xa = (1 - =1) S [1 50 1(x+1)² -1/2
Proof. Combine Lemma 3 with Lemma
Corollary 6 (Case when E (1,2 For the polynomials pı (A) = - Pz(A) = 15 +
8
8 x2, the residual recursion specializes to de.ji1 < 87, and 8,
j+1 S 8% for all br,j E [0, 1). These
are concrete instances of the general residual map in Lemma
-1-(1 (1 (1-u) pp) (1 w) gives 8,41 = (3 4
Proof. Direct substitution into ((u) = 1
< 83 since
3 8 < 4 on [0, 1] for 1. Also, 8,-1 61100+155+995 < 63 since 40 + 155 + 962 < 64 on
[0, for 2 (check 8 = 1).
35

---

Published as a conference paper at ICLR 2026
D.1 DETAILED PROOFS FOR CASE WHEN {1,2}
Residual Decay by Ist-order NEWTON-SCHULZ Polynomial. Define the polynomial p()) for NEWTON-SCHULZ steps
1 2 2
p(A) マ
Then for any fixed and any 2 O, the residual òr.j satisfies
8t,j-1 S (dr,)²
Consequently, if 6,,0 & p < 1 sufficiently small for all t, then òng S p²" for all t.
Proof. Deriving the function T(A) in Lemma
2
3 1
3
1 4
T(A) xip(A)] =
2
2
4
x
= +
2
1
Then the derivative of T(A) is
3
3
(A) = 4
31 +
4 4 (A
x2
1)(2 -3)
3)
Since 7A()) 2 O for À E [0. and r(1) = 1, we can apply Lemma و.j is updated by one NEWTON-SCHULZ. Step as
The orthogonality residual
8rj+1 = ((6r.) = 1 - (1 -1- 1-),))(((( dr,s)l?
3
(1 - -81.)) (2,(++ s 87
- (1-(1-1 5r,5)
2
By induction, after steps, we get the relationship, dzg S (dr.0)²'. Given the assumption that o.p S p < 1 for all f, the conclusion follows directly:
δe.q S p²"
This shows that the orthogonality residual decreases quadratically with each iteration, which is very rapid rate of convergence.
Residual Decay by 2nd-order NEWTON-SCHULZ Polynomial. Define the polynomial p(A) for NEWTON-SCHULLZ steps as
15
5 3 x+ 8
P(A)
2
Then for any fixed and any 0, the residual de,j satisfies
8tgz11 S (ōt,)²
Consequently, if 6,,0 S < 1 is sufficiently small for all t, then de, S pd for all t.
Proof. Deriving the function r(A) in Lemma 2
15
5
3
2
T(A) \1p()))² A
+
x2
8
4
8
Then the derivative of 7(A)
15
5
3
15
5 \ +
3
((A)
1
+ 21
12
8
15
5
3
15
3
3 2
1
(20
(5
32)3) (4 (1 (1+ x)2)
64
36

---

Published as a conference paper at ICLR 2026
Since (2) 2 0 for À E [0, and r(1) 1, we can apply Lemma 2
The orthogonality residual
jig is updated by one NEWTON-SCHIIN step as
81.+1 = x(b(,)) = 1 - (1
51,5 Dp((1 - 6c,5)1
15
-.r,)) + (1 )(,)2 2
1 - (1 ò1.5)
(1
1
3
1 (1 )(z)
+
+
57
2
8
3 0
9
1 81.)
1+ 1+6,, 87, +
0is
2, (40 156,,
+ +96%
< 6%,
64
By induction, after q steps, we get the relationship, dup S p < 1 for all f, the conclusion follows directly:
deg S (6r,0) Given the assumption that
3
This shows that the orthogonality residual decreases cubically with each iteration, which is very rapid rate of convergence.
37

---

Published as a conference paper at ICLR 2026
E WALL-CLOCK VIA COMPUTATIONAL COMPLEXITY.
At each iteration i, MUON performs an orthogonalization of the momentum matrix M, via either SVD NEWTON-SCHULZ (NS). We write & gemm for the effective GEMM throughput (FLOP/s), and Pevd for the effective throughput of the SVD routine. In practice & gemm > 2sv due to far higher hardware utilization of GEMM.
Per-iteration Orthogonalization FLOPs. For single layer index by & with m S n (for m > n, apply to M, and transpose-trick):
MUON with SVD. A thin SVD of M, E mxxx and extracting the polar factor U.V2" costs (Golub & Reinsch 1971:
FLOPs and (m, n)) @4m + 8m²)
Wall-clock time per layer svd FLOPS (9)
svd (m, n)/B.w
MUON with NS (g-steps, k-degree). NEWTON- -SHTUL- follows Horner's rule when cursively updating the scaled momentum matrix using the NEWTON-SCHUIL polynomial. NEWTON-SCHUL.Z forms A XX R=xm and applies the degree-x polynomial to X via Horner's rule. Each NS step needs one m x n by n x m GEMM to build A and multiplies AY (each x by m n1). Hence.
FLOPs (P) nss (m, n.; q,;q,K) = O(2q(K + 1)m²n)
Wall-clock time per layer Lss = FLOPS (m, k)/$ gemm-
gemm-
Lemma 11. For layer with m S n, the wall-clock time ratio between MUON with SVD and MUON with NEWTON-SCHE SCHULL (q-steps, n-degree) is,
(4m2 8m
m 7
evd
gonam
gemon
(2g((x + 1)m2n)
4 svd
a(K +1
O svv
efficioncy natio %0
Discussion of Lemma 11 With the practical setting q {2,3) and e {1,2), the algebraic factor 2+4(m/n) [(++1) is (0.3~~1), so the wall-clock speedup is essentially the GEMM/SVD efficiency ratio. On modern GPUs, & gemmm /Psd is often 4~10, so NS typically yields multi-x speedup per iteration over SVD, matching empirical observations in Figure
Practical Interpretation. NEWTON- COWTON-SCHULZ SCHULZ scales linearly in (accuracy knob) and uses only GEMMs, which map efficiently to GPUs. Exact SVD pays an additional m² term and typically incurs larger constants. Hence for small y and modest x, NEWTON-SCHULZ is substantially cheaper update than a full SVD while achieving near-exact orthogonalization in practice.
38

---

Published as a conference paper at ICLR 2026
F NUMERICAL EXPERIMENTS DETAIL
F.1 EXPERIMENTAL SETTING
Task and metric. All experiments are conducted on CIFAR-10 (50k train 10k test) with the standard channel-wise normalization mean (0.4914, 0.4822, 0.4465) and std (0.2470, 0.2435, 0.2616).
We report cross-entropy train loss and test loss. Plots show the (mean = 1 std) over 5 independent runs, and we also report the wall-clock time (in seconds) accumulated from the beginning of training.
Model. We use compact convolutional network (CIFARNET, approximately 2M parameters): fixed whitening 2 convolution (weights frozen, bias trainable), followed by three ConvGroups (each: 3 x conv MaxPool2d(2) - GELU + 3 3 conv GELU) with widths (64, 256, 256) and linear head. BatchNorm layers keep affine weights frozen (biases are trainable). For stability, the first portion of each convolution is Dirac-initialized, and the linear head is variance-normalized.
Hy Hyperparameters and schedules. Unless stated otherwise, we train for 50 epochs with a batch size of B 512 on the GPU (the CPU fallback uses B 256). Each run uses different seed.
Main optimizer (one of MUON (g), MUON (SVD), or SGD-M) on convolutional filters: learning rate 1) = 0.0860632, momentum / = 0.730778.
Auxiliary SGD (whitening bias, BatchNorm biases, linear head): learning rates Yother 1.4949 x 10-3 and Thead = 1.72446 with momentum 0.989703 (Nesterov).
Label smoothing 0.2, and gradient clipping || $ 1.0.
Schedule: The same warm-up cosine schedule applied to all optimizers: a 5% linear warm-up of total steps followed by cosine decay to O. Schedulers step once per update.
Evaluation. At the end of each epoch, we evaluate the test loss using the same normalization (no augmentation). We record (i) epoch-wise training loss, (ii) test loss, and (iii) the cumulative wall- clock time. Results are aggregated across rums. Figures report (mean + 1 std) and include both epoch-aligned and time-aligned views disentangle statistical and systemic effects.
F.2 NEWTON-SCHULAN STEPS (g) ABLATIONS.
Optimizers under comparison. We compare:
1. MUON q-step: SGD with momentum followed by an orthogonalization of the momentum matrix using q NEWTON-SCHUUL steps (Algorithm []: E 10,1, The case q 0 is normalization-only ablation (no orthogonalization)
2. MUON (SVD): MUON with an exact polar step UVT via SVD.
3. SGD-M: SGD with momentum baseline with identical schedules.
All methods share identical schedules and auxiliary updates (Appendix F.I
Orthogonalization details.
For MUON with g-NEWTON-SCHULZ steps, we use
NEWTON-SCHULA polynomial, Pr.
p()) a +
(a, 3/44,
applied as X + ax + (bA+ + cAP) with A X and X = M/]] if the matrix tall, we employ the transpose trick. The SVD variant normalizes M by its Frobenius norm and returns UV all MUON variants, each weight tensor is re-normalized once per step to the Frobenius norm / nut__chancels to stabilize scales.
Ablations on q. Our main comparison varies the number of DEWTON-SCHUT steps q E (1,2,3) while holding the polynomial fixed to Pa. and contrasts these with MUON with SVD and SGD-M. All other components (architecture, schedules, augmentation) are kept identical across methods to ensure fair comparison.
39

---

Published as a conference paper at ICLR 2026
G ADDITIONAL NUMERICAL EXPERIMENTS
Optimizers compared:
SGD with Momentum
Muon (Newton-Schulz) with 2, steps per update
Muon with exact SVD (polar factor)
Dataset and Model:
MNIST (60K) MIPP (0.5M)
CIFAR-IO (50K) CifarNet (2M) Main text
CIFAR-100 (50K) ResNet-18 (11.2M)
Tiny-ImageNet (100K) / WideResNet-28-10 (36.6M)
Web (10M tokens from sample-10BT) NanoGPT (124.2M) & GPT-2 (1.3B)
block size 1024
num blocks = (10M - 1) // 1024 9,765 sequences (for causal LM)
nllaer:: 12 & 24 (transformer blocks)
head: 12 & 16
emmb:dd: 768 & 2048
Vocab size: tokenizer.vocab size from GPT2TokenizerFast (50257)
Token embedding: Nn.Embedding(ooabb size, 768)
Positional embedding: www.Eeeeddnng(blocckkk.. _ize, 768)
Each of the 12 transformer blocks
* LayerNorm - multi-head causal self-attention (QKV + output projection) 7 residual
LayerNorm - 4x-wide MLP (3072 hidden) residual
Final LayerNorm
Total n_params: 124.2M 1313.63M (1.31B)
Hyper-parameters
MLP on MNIST:
model: 784 - 512+ 256 + 10
learning rate: 0.08; momentum: 0.7
256 batch size; 50 epochs, 5 runs
ResNet-18 18 on CIFAR-100:
model: torchvision. models.resneti learning rate: 0.08; momentum: 0.7
512 batch size: 50 epochs, 5 runs
WideResNNet-28-10 on Imag-Nn:::
model: WideRResNer(dephh288, widen_facton factor=10, num clm_clases==200 r00p_rate=0.0)
learning rate: 0.08; momentum: 0.7
128 batch size: 30 epochs, 3 runs
NanoGPT & GPT-1.3B on FineWeb:
model: NanoGPT GPT-1.3B
learning rate: 0.02; momentum: 0.95; batch size:
10M training tokens, 1M validation tokens
8 RTX-3090 GPUs
max steps: 6000
40

---

Published as a conference paper at ICLR 2026
G.1 MLP ON MNIST
MNIST/ MLP Muon(q=1)
Muon(q=2)
Muon(q=3)
Train Loss vs. Wall Clock Time
Train Loss vs. Epach
0.64
0.64
0.62
0.60
Train
0.54
0.52
0.52
0.50
0.50
50s
150s
25
30
35
Wall-Clack
Figure 2: Train losses of MLP MNIST across wall-clock time and epochs
Table 2: Wall-clock time training performance of MLP (0.5M) MNIST dataset
Wall-clock time train loss
Optimizer
sec
sec
150 scc
200 see
see
SGD-M
0.550
0.525
0.517
0.514
0.513
Muon with SVD
0.616
0.612
0.600
0.583
0.565
Muon (g=1)
0.526
0.514
0.506
0.502
0.501
Muon (q=2)
0.532
0.518
0.509
0.503
0.501
Muon (g=3)
0.548
0.529
0.511
0.503
0.501
G.2
CIFARNET ON CIFAR-10
CIFAR- 10 CifarNet Muon(q=1)
Muon(q-2)
Muon(q-3)
Train Loss vs. Wall-Clock Time
Train Loss vs. Epoch
1.4
1.4
1.3
1.3
Loss
Train Loss
1.2
1.2
1.1
1.1
1.0
1.0-
0.9
0.9
30s
40s
60s
76s
Wall-Clock Time
Epoch
Figure 3: Train losses of CifarNet on CIFAR-10 across wall-clock time and epochs
Table 3: Wall-clock time training performance of CifarNet (2M) on 10
Wall-clock time train
Optimizer
10 sec
20 sec
30 sec
40
60 sec
SGD-M
1.366
1.157
1.084
1.045
1.021
1.010
Muon with SVD
2.211
1.334
1.227
1.207
1.193
1.182
Muon (q=1)
1.130
1.014
0.945
0.905
0.888
0.884
Muon (g=2)
1.129
1.064
1.001
0.932
0.888
0.876
Muon (g=3)
1.191
1.145
1.079
0.990
0.905
0.876
41

---

Published as a conference paper at ICLR 2026
G.3
RESNET-18 ON CIFAR-100
CIFAR-100 ResNet-18
Muon(q=1)
Muon(q=2)
Muon(q=3)
Train Loss v5. Wall Clock Time
Train Loss vs. Epach
2.8
2.8
2.6
2.6
2.4
2.4
2.0
1.8
1.8-
1.6
1.6
Wall-Clock
Epoch
Figure 4:
Train losses of ResNet- 18 CIFAR-100 across wall-clock time and epochs
Table 4: Wall-clock time training performance of ResNet- 18 (11.2M) CIFAR-100
Wall-clock time train loss
Optimizer
sec
200 sec
300 see
see
500 sec
SGD-M
3,096
2.612
2.324
2.157
2.072
Muon with SVD
3.202
2.767
2.668
2.594
2.522
Muon (q=1)
2.427
1.986
1.716
1.563
1.514
Muon (g=2)
2.407
2.127
1.826
1.584
1.495
Muon (g=3)
2.554
2.329
1.985
1.624
1.481
G.4 WIDERESNEA-28-10 ON TINY-IMAGNNET
Tiny- Tmy-ImmaeNNet-200 Muon(q-1)
/WideRes
Muon(q-2)
Muon(q-3)
Train Loss vs. Wall-Clock Time
Train Less vs. Epoch
5.0
5.0
4.5
4.5
4.0
4.0
3.5
3.5
3.0
3.0
2.5
25
2.0
05
250s
500s
750:
1000s 1250s 1500: 1750s 2000s
Wall-Clock Time
Epoch
Figure 5: Train losses of WidcRessNet22-110 on Tiny-ImageNet across wall-clock time and epochs
Table 5: Wall-clock time training performance of WideResNet-28- (36.6M) on Tmny-ImageNet
Wall-clock time train
Optimizer
400 sec
800 sec
1200 sec
1600 sec
2000 sec
SGD-M
4.694
4.299
4,045
3.914
3.857
Muon with SVD
4.267
4.020
3,846
3.734
3,520
Muon (g=1)
4.035
3.369
2,903
2.495
2.144
Muon (g=2)
3.766
3.246
2,815
2.311
1.856
Muon (g=3)
3.666
3.317
2.929
2.372
1.800
42

---

Published as a conference paper at ICLR 2026
G.5
NANOGPT ON FINEWEB
FineWeb NanoGPT Muon(q=1)
Muon(q=2)
Muon(q=3)
Wall-Clock Train Loss vz. Wall Clock Time
Train Loss vs. Iterationn
10
10-
0
Train
Train
200s Wall-Clock Time
5600
Figure 6: Train losses of NanoGPT on Web across wall-clock time and epochs
Table 6: Wall-clock time training performance of NanoGPT (124M) FineWeb
Wall-clock time train loss
Optimizer
sec
200 see
see
400 sec
500 sec
see
SGD-M
10.938
7,656
6.000
5.375
4.938
4.594
Muon with SVD
10.250
9.188
8.625
8.062
7.594
7.125
Muon (g=1)
5.969
4.094
2.469
1.367
0.879
0.637
Muon (g=2)
5.000
2.344
1.047
0.395
0.157
0.030
Muon (g=3)
3.984
2.125
1.180
0.523
0.142
0.026
G.6 GPT-2 BASED MODEL (1.3B) ON FINEWER
FineWeb / GPT-2 based model (1.38)
SGD-M
Muon(5VD) - Muon(q-1) Muon(q-2)
Muon(q-3)
Train Loss Wall-Clock Time
Train Loss Step
Train Loss
Train
06
500s
1000s
1500s
2000s
2500s
3000s
3500s
600
1200 1800 2400 3000 3600 4200 4800 5400 6000 Iteration
Wall Clock Time (s)
Figure 7: Train losses of GPT-2 based model (I.3B) on FineWeb across wall-clock time and epochs
Table 7: Wall-clock time training performance of GPT-2 based model (1.3B) on FineWeb
Wall-clock time train loss
Optimizer
600 sec
1200 sec
1800 sec
2400 sec
3000 sec
3600 sec
SGD-M
6.4062
5.3750
4,2812
3.2969
2.4219
2,0156
Muon with SVD
8.2280
8.0810
7,9340
7.7870
7.6400
7.4930
Muon (g=1)
6.0625
3.5156
1.0781
0.1436
0.1338
0.0109
Muon 2x=
5.1562
2.8906
0,5898
0.1133
0.0349
0.0113
Muon (g=3)
5.6562
3.2031
1.2188
0,4609
0.1201
0.0286
43

---

Published as a conference paper at ICLR 2026
H
ADDITIONAL ABLATION EXPERIMENTS
H.1
NEWTON--SCHUI--POOMXMMAAA DEGREE-K ABLATIONS.
K-2
Train Loss Epoch
Test Loss vs. Epoch
0.98
0.50
0.94
0.48
0.92
0.46
0.42
0.88
0.40
16
20
28
32
36
16
20
28
32
36
Epach
Epech
Train vs.
0.54
0.52
0.96
0.48
10
0.92
0.46
0.44
0.90
0.40
205
305
40s
50s
60s
70s
205
305
50s Wall Time (s)
40s
60s
70
Wall-Clock Time (s)
Figure 8: Degree 6 sweep. NEWTON-SCHULZ polynomial degree E (l,....5) 5) at fixed 3. Larger improves train/test loss but increases time per step.
Beyond the step-sweep in Fig. we perform controlled ablation that varies the degree к of the NEWTON-Sctn SCHULZ polynomial while fixing the number of NEWTON-SCHULZ steps to 3 for all variants. Concretely, we instantiate a family of MUON with degree-x polynomial optimizers whose orthogonalization step applies, per layer, the update
M
X Pre (XX XX, X
|M]]
where M is the momentum matrix, and Px. (A) = Dm Am is the degree-K NEWTON-SCHULZ polynomial that matches the derivatives of -1/2 at x up to order The coefficients are generated analytically from the Taylor series of X - at 1, ensuring the residual recursion 0/41 S (6,)*+1 (Lemma 3). Tall matrices are handled using the transpose trick.
3, increasing the degree (loss drops faster at a fixed epoch) but lengthens each step. yielding a clear accuracy-time trade-off (see Fig. 8) (This mirrors the theory that the residual contracts dJ+1 87 (Lemma 3 while computation scales with polynomial evaluations.)
Ablation on degree K. Fixing
{1....,5] improves optimization
Compared optimizers. We evaluate {1,2,3,4 4, 5) under fixed iteration budget q 3. This ablation isolates the effect of the polynomial degree x at fixed step q, directly testing the theory- driven prediction that larger x accelerates orthogonality residual decay (see Table Tp.
44

---

Published as a conference paper at ICLR 2026
H.2 RANK-DEPENDENCE.
SGD-M
(SVD(SVDD)
Muon(NS)
Raw scaling (log-log)
Normalized by v7
x10
10
avp. PaAWIn JVT per epoch
0.20
10
0.16
62
0.10
avg. |9//W]
-022
10
-040
-0.11
10
16
128
216
16
128
216
r- m-mmin,mmn n)
r= m-minmm.m n)
Figure 9: Rank dependence. Epoch-averaged gradient norm vs. rank (left: raw log-log; right: normalized by vF). MUON variants are nearly r-invariant; SGD-M scales up with rank.
Table 8: log-log slopes (w) from Fig.
Method
(raw)
w (normalized by vr)
SGD-M
0.292
-0.208
MUON (SVD)
0.102
-0.398
MUON (NS)
-0.106
-0.606
Rank dependence. We vary the monitored layer's effective rank E {16, 32, 64, 128, 216} and plot epoch-averaged | [W) on a log-log scale. SGD-M shows positive slope a 0.3 (grows with r), whereas MUON and its variants nearly flat. After normalizing by vr. the two MUON variants follow the predicted 11/ trend, while SGD-M remains non-flat (see Fig. 9).
Goal and prediction. We test how the nuclear-norm the per-step gradient on monitored layer scales with the layer's dimension parameter := min{ mi, n) for weight W E mR The theory predicts that orthogonalizing the momentum suppresses the r-growth which occurs in either MUON with NEWTON-SCHULZ MUON with SVD, while SGD-M has // dependence.
Controlling r. We vary the out-channels of the first convolution in the first ConvGroup and keep all other widths fixed. For this layer, the weight tensor has shape |out, in, 3,3] with in 24 (from the fixed "whitening" 2 x conv). Flattening by rows yields a matrix of shape m x n with m = out minfout out, 216}. We sweep out {16, 32, 64, 128, 216),
and 3 24 9 9- 216, hence E {16, 32, 64, 128, 216].
Optimizers under comparison. We compare:
. SGD-M (baseline),
MUON (SVD) (exact polar UVT), ), and
MUON (NS) with = Newton-Seen SShnnz steps.
All methods share identical schedules and auxiliary updates (Appendix F.I.
Results. Across E {16, 32, 64, 128, 216}. SGD-M exhibits a positive log-log slope (about 0.3), while both MUON variants are nearly flat. After dividing by vT, the MUON curves show a slope of about -0.5 (MUON with SVD: -0.4 and MUON with NEWTON-SCHUIE -0.61). as predicted, whereas SGD-M becomes almost flat (slope with -0.21).
45

---

Published as a conference paper at ICLR 2026
H.3
BATCH SIZE B ABLATIONS.
We sweep B E (64, 128, 256, 512, 1024) with MUON (g= 3) under identical schedules and report ppoch-aligned and time-aligned views (Fig. TO.. The schedule is step-based, larger B implies fewer total steps over E epochs. We report both epoch-aligned and wall-clock-aligned curves.
From systems perspective, increasing B improves throughput up to regime of diminishing turns. From an optimization perspective, a larger B reduces gradient noise but also decreases the frequency of orthogonalization steps per epoch (Nproj Nuain/ B).
In our runs, the best -oac-ua-yyy is achieved for large batch size B = 1024, while small B suffers from noise, taking a greater amount of time.
B=64
B=128
B=256
B=512
B=1024
Train Loss vs. Epoch
Loss vs. Epoch
2.25
2.2
2.00
2.0
1.75
Test Loss
1.50
1.6
1.25
1.4
1.00
10
0.50
12
18
24
30
36
42
48
12
18
24
30
36
42
48
Epoch
Epoch
Train Loss vS. Wall-Clock Time
Test Loss vs. Wall-Clock Time
2.25
2.00
2.0
1.8
1.50
1.25
1.4
1.00
1.2
0.75
100s
300s
400s
(s)
Wall-Clock
Figure 10: Batch size. Train/test loss epoch and wall-clock for B E (64, 128, 256, 512, 1024].
46

---

Published as a conference paper at ICLR 2026
H.4
2 NS POLYNOMIAL VS. AD-HOC 2 NS POLYNOMIAL
Ad hoc Potynomial (degree 2)
NS Pelynomia idegree
Train Epoch
Loss Epoch
115-
1.3
2.00
2.55
Epach
135
385
1.0
2.00
1.00
36
15
Val-Clecc Time
Wal-Clece Time
Figure 11: NS polynomial vs. Ad-hoc polynomial. Train/test loss vs. epoch and wall-clock.
Analysis of Pad-hoc
Let Pad-hoc (A) = 3.4445 - 4.7750 A + 2.0315 X and T(A) = Ap(A)²
The statement that Tad-hoc (A) is monotone non-decreasing on [0, 1] is false, and the underlying con- dition that Pas-too (A) E [0, 1] is also not met.
On the interval [0, 1]. the range of p()) is (0.701,3.4445) Since this range is not contained within [0, 1]. the premise p(A) E [0, 1] is false.
function is monotone non-decreasing if its derivative is greater than or equal to zero over the entire interval. The derivative of TA)) is:
d
Tul-hhoe (A)
Tad huc (A) = 20.635A - 77.5824X + 110.3556X - 65.781A + 11.8641
dX To check for monotonicity, we can evaluate the derivative at the endpoints of the interval [0, 1]:
* At À = 0: Toe- (0) = 11.8641
At À = 1: Tac-hoc (1) = 20.635 - 77.5824 + 110.3556 - 65.781 + 11.8641 = -0.5087
Since hod (0) > O and T buc (1) < 0. the derivative changes from positive to negative within the interval. This means the function Fad-boe (X) increases for a portion of the interval and then decreases.
Therefore, the claim that Tad-boo (A) is monotone non-decreasing on [0, 1] is false. The function has
local maximum approximately À As 0.308
Hence, the monotonicity premise of Lemma fails for Pud-hoc, even though 7(1) pp1)² 0.701 < 1 holds. This explains why our guarantees apply to pr. but not the ad-hoc quadratic, which remains an empirical heuristic.
47


Reviews:
[{"review_id": "ys6gLs0d6n", "reviewer": "Reviewer_Wnwj", "summary": "In this paper, the convergence of Muon is analysed, for both the original and derivative versions. It is proved that Muon with Newton-Schulz converges with the same rate as SVD-polar idealization, and it can remove the square-root-of-rank loss compared to vector-based optimizers. The results explain the performance gain of Muon compared to vector-based optimizers, and are validated with experiments on the CIFAR-10 dataset and a CNN model.", "questions": "None.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This is one of the first convergence analysis result for Muon with finite steps of Newton-Schulz, which is a big step forward to narrow the gap between theory and practice.", "weaknesses": "The experimental part is relatively too simple. Experiments with larger and various datasets, larger number of parameters in the model, various model types such as MLP and transformer, more epochs, and more specific analysis how the results validate the theoretical analysis, will be nice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GRutfv1Goe", "reviewer": "Reviewer_WRnz", "summary": "The paper investigates the convergence properties of the MUON algorithm, which demonstrates its superior properties against SGD or ADAM in terms of convergence speed. Other than existing works with an exact SVD orthogonalization of the momentum in its native matrix form, the paper studies the MUON with a finite steps of NEWTON–SCHULZ. The main results show that the MUON algorithm with a fixed finite steps of NEWTON–SCHULZ converges a stationary point with the same rate as the exact SVD-orthogonalization, up to a constant factor. The work further shows that this constant factor decays doubly exponentially in the fixed finite step of NEWTON–SCHULZ $q$ and improves with the degree of a polynomial used in NEWTON–SCHULZ $κ$.", "questions": "For a given setting of training a DNN, how close are the stationary points generated by the MUONs with finite NEWTON–SCHULZ and the exact SVD? If they are observed to be only satisfying the $\\epsilon$ stationary point, are the qualities of these stationary points similar, in terms of generalizability to unseen data?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The paper addresses a very demanding question about the convergence speed of the recently proposed MUON algorithm with a finite step of NEWTON–SCHULZ. As an approximate version of the MUON with the exact SVD orthogonalization of the momentum update, the presented analysis provides a solid theoretical understanding of the convergence behavior of the MUON in connection to its exact SVD counterpart, and ultimately guarantees the properties of deploying the MUON algorithm with a finite step of NEWTON–SCHULZ to large-scale training of DNNs, such as LLMs.", "weaknesses": "The core message conveyed in the paper was to show that the MUON algorithm with a finite step of NEWTON–SCHULZ converges similarly to some stationary point as its exact SVD counterpart in terms of convergence rate. However, one crucial aspect of training DNNs is the quality of stationary points. In other words, the discrepancy between stationary points generated by the MUONs with finite NEWTON–SCHULZ and the exact SVD can be large, thus is worth investigating.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9K8pd1V3Ur", "reviewer": "Reviewer_KDNU", "summary": "The goal of this work is to provide a formal analysis and proof of the convergence of the recently introduced and potentially useful MUON optimization algorithm using momentum orthogonalization with a few newton-steps. This work fills the gap between the existing theory on MUON with an exact SVD-based polar factor. My main concern is that MUON is not widely used, with mostly a few arXiv papers referenced for its use. \n\nUsing standard analytic assumptions, tools and techniques, the authors provide complete convergence rates (dependence on the standard terms) for SGD with momentum, MUON with SVD, and MUON. Diverse numerical experiments on CIFAR-10 with modest sized CNN predominantly support their theory. Their results explain several contrasts between MUON w/SVD,MUON, and SGD w/momentum, and closes the theory-practice gap.", "questions": "Typos:\nL53: ‘much’ -> ‘large’\nL80: remove comma after ‘Shultz’\nL84: ‘closer’-> ‘close’\nL116: include ‘in “the” MUON…”\nL293: ‘Decaying’ -> Decay", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "This paper is very clear and generally very well written.\nThe numerical results are extensive, but primarily in the appendix. I suggest moving several of the key plots from the appendix to the main text.", "weaknesses": "I did find many redundant statements of the main results throughout the paper (often with exact wording…); while I generally agree that repetition like this can enhance clarity of interpretation and impact, I found this manuscript to be excessive in that regard. The numerical results are extensive, but primarily in the appendix. I suggest moving several of the key plots from the appendix to the main text. Also, from what I saw, there is little additional information provided by the presentation of both Loss and Accuracy, so I suggest the authors pick one for the main text and move the other quantification to the Appendix. A bit more explanation of the details of the results presented in Figure 1 w.r.t. Learning dynamics across values of ‘q’ would be beneficial. A few sentences on mathematical definition and interpretation of the different norms used (Section 3.1) would be helpful for a broader audience. The definition of ‘r’on line 151 is very far removed from its first use on line 280; suggest moving. Remove variables from the abstract.\n\nMy", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NQEM2u9PeV", "reviewer": "Reviewer_V23p", "summary": "This paper studies the MUON optimizer, which is designed for matrix-structured parameters. In practice, MUON replaces expensive SVD computations with a few Newton-Schulz (NS) steps to orthogonalize its momentum. Previous analyses only covered the idealized SVD version. This paper gives the first convergence proof for the practical version that actually uses NS.\n\nThe main result shows that MUON with NS converges to a stationary point at the same rate as the SVD version, up to a small constant that converges doubly exponentially fast to 1 as the number of NS steps increases. The paper also shows that MUON improves rank dependence compared to SGD with momentum. Experiments on CIFAR-10 confirm the theoretical findings.", "questions": "See weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well written and well organized. The intuition behind the proofs is explained in a way that makes the technical results easy to follow.\n\nThis work provides the first convergence proof for MUON when the orthogonalization step is done with Newton–Schulz iterations. This directly corresponds to how MUON is implemented and used in practice, rather than analyzing an idealized or simplified variant. This is an important contribution because most previous works relied on the unrealistic assumption of an exact SVD-based orthogonalization, which is not used in real training.\n\nThe theoretical assumptions are reasonable and consistent with standard conditions in stochastic optimization. They are stated clearly and make sense for modern deep learning models. It is also appreciated that the authors do not assume extra smoothness or artificial conditions beyond what is standard for first-order methods.\n\nThe main theorems are precise, and the proofs are well-structured and clear. The paper shows in detail how the Newton-Schulz approximation error decreases and how it influences the convergence rate. The results are intuitive and easy to interpret.\n\nFinally, the comparisons between MUON with NS, MUON with SVD, and SGD with momentum are well designed. They are based on the same assumptions, metrics, and convergence definitions, which makes the differences in behavior easy to understand. This unified view helps highlight the benefit of MUON’s matrix-aware design.", "weaknesses": "The paper is already strong in both quality and presentation, but there are still a few limitations.\n\nThe numerical experiments are pretty limited (which is fine given how popular Muon is), but it would be better to have larger-scale experiments when comparing methods. The results are shown only on CIFAR-10 using a small CNN. While this setup is enough to confirm the theory, it does not demonstrate the behavior of MUON in more realistic large-scale scenarios. Testing the optimizer on larger architectures, such as Transformers or vision models with high-rank layers, would make the empirical section much more convincing and highlight the practical impact of the theoretical results.\n\nAlthough the theoretical assumptions are standard and well-justified, their validity in large deep networks can be questioned. In practice, smoothness and bounded variance often fail to hold exactly. For example, Transformer gradients can have heavy-tailed distributions, which makes the variance assumption only approximately true.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)