A Unified Theory of Quantum Neural Network Loss Landscapes
Authors: Eric Ricardo Anschuetz
Venue: ICLR 2025 Poster (2025)
Keywords: quantum machine learning
neural tangent kernel
loss landscape
spin glass
Kac–Rice formula
Primary area: optimization
TL;DR: We unify previous results on the training behavior of quantum neural networks by proving they asymptotically behave as a novel class of random processes.

Abstract: Classical neural networks with random initialization famously behave as Gaussian processes in the limit of many neurons, which allows one to completely characterize their training and generalization behavior. No such general understanding exists for quantum neural networks (QNNs), which—outside of certain special cases—are known to not behave as Gaussian processes when randomly initialized. We here prove that QNNs and their first two derivatives instead generally form what we call "Wishart processes," where certain algebraic properties of the network determine the hyperparameters of the process. This Wishart process description allows us to, for the first time: give necessary and sufficient conditions for a QNN architecture to have a Gaussian process limit; calculate the full gradient distribution, generalizing previously known barren plateau results; and calculate the local minima distribution of algebraically constrained QNNs. Our unified framework suggests a certain simple operational definition for the "trainability" of a given QNN model using a newly introduced, experimentally accessible quantity we call the "degrees of freedom" of the network architecture.

Published as a conference paper at ICLR 2025
A UNIFIED THEORY OF QUANTUM NEURAL NETWORK LOss LANDSCAPES
Eric R. Anschuetz
Institute for Quantum Information and Matter & Walter Burke Institute for Theoretical Physics, Caltech Pasadena, CA 91 91125, USA
ABSTRACT
Classical neural networks with random initialization famously behave as Gaussian processes the limit of many neurons, which allows one to completely charac- terize their training and generalization behavior. No such general understanding exists for quantum neural networks (QNNs), which-outside of certain special cases- are known to not behave Gaussian processes when randomly initial- ized. We here prove that QNNs and their first two derivatives instead generally form what we call Wishart processes, where certain algebraic properties of the network determine the hyperparameters of the process. This Wishart process de- scription allows us to, for the first time: give necessary and sufficient conditions for QNN architecture to have a Gaussian process limit; calculate the full gradi- ent distribution, generalizing previously known barren plateau results; and calcu- late the local minima distribution of algebraically constrained QNNs. Our unified framework suggests a certain simple operational definition for the "trainability" of a given QNN model using a newly introduced, experimentally accessible quantity we call the degrees of freedom of the network architecture.
INTRODUCTION
1.1 MOTIVATION
One of the miracles of machine learning on classical computers that simple, gradient- opti- mizers can efficiently find the minimum of extremely high-dimensional, nonconvex loss landscapes, allowing for the efficient training of deep neural networks. Over the past decade this has been un- derstood in more and more detail via random matrix theory. In particular, it is now known that the loss landscapes of randomly initialized, wide neural networks are distributed as Gaussian processes with covariance given by the so-called neural tangent kernel (NTK) (Ncal 1996; Choromanska et al. 2015 Chaudhari] 2018 Lee et af.) 2018). The NTK is completely determined by the neural structure of the network, linking the asymptotic behavior of the network to architectural choices made in its construction. This understanding of classical neural networks has been used to show that wide neural networks train efficiently via gradient descent (Choromanska al. 2015 Chaudhari 2018 Jacot et al. 2018 Allen et al. 2019). It also explains other emergent phenomena in deep learning such as the remarkably good generalization performance of neural networks beyond what learning theory predicts (Jacot et al. 2018 Wei et al. 2022).
One might hope that similar, universal story would exist for quantum neural networks (QNNs). These are classes of neural networks where the associated loss function & takes as input quantum state p > N with trace 1, conjugates p by unitary operation Uo E U (N) parameterized by 0 E R.P and then takes the Frobenius inner product with the Hermitian O E CNNN, That is,
é (p: = Tr -T (pU)OU.).
Such networks are defined by the choice of parameterization for Ue (typically called the ansatz) and the Hermitian O (here called the objective observable) (Peruzzo al.) 2014). and are known to efficiently (on a quantum computer) perform learning tasks provably difficult using traditional machine learning methods (Liu et al. 2021 Hastings & O'Donnell 2022 Anschuetz al. 2023b

---

Published as a conference paper at ICLR 2025
Huang et al.
2024
Anschuetz & Gao. 2024). Unfortunately, it is known that QNNs generally do
not have a Gaussian process (or quantion neural tangent kemel (QNTK)) asymptotic limit (Gareía)
Martin et al. 2023 Girardi & Palmal 2024). Indeed, it is not even obvious whether equivalently simple universal description exists due to QNN training dynamics differing completely in various parameter regimes. For instance, the loss landscapes of generic shallow QNNs are known to be described by so-called Wishart hypertoroidal random fields (WHRFs) and dominated by poor local minima (Anschuetz] [022; Anschuetz & Kiani| 2022); this is far cry from the effectively convex training landscapes of deep quantum networks which, in certain circumstances, can be shown to have a Gaussian process limit (Liu et al. 2022; 2023) You al., [022] Garcia-Martin et al., [023] Girardi & Palma 2024; García-Martin et al. 2024). Unfortunately, these deep networks are still untrainable in practice due to an exponential decay in their gradients known as the barren plateau
phenomenon (McClean et al. 2018), making estimating gradients on a quantum computer asymp-
totically intractable!
The presence of barren plateaus or poor local minima in QNN loss landscapes paints pessimistic picture for the practical utility of generic QNNs. However, these negative results can be circum- vented by considering structured QNNs, further complicating any unifying theory of the asymptotic training behavior of QNNs. For instance, Ug may be constrained to belong to some low- www-dimensional Lie subgroup G of the full unitary group, and O in the generating algebra of G. This is known as the Lie algebra-supported ansatz (LASA) setting, and has been shown to be capable of preventing the barren plateau phenomenon from occurring (Fontana al., 2024 Ragone et al., (022). Though many of the initial proposals for such structured networks have since been "dequantized" (Anschuetz al. 2023a Goh et al. 2023 Cerezo et al., 2023) i... efficient classical algorithms have been found which simulate such networks there still do exist unconditionally provable quantum advan- tages in using such networks for certain learning tasks (Anschuetz et al., 2023b] /Anschuetz & Gao) 2024). Though this is perhaps the most promising direction in QNN research, nothing concretely is known about the loss landscapes of such networks beyond the variance of the loss function over parameter space.
1.2 CONTRIBUTIONS
Fully understanding how the various phenomenologies of QNN loss landscapes are related is im- portant if we ever hope have as deep an understanding of QNNs as the neural tangent kernel has enabled for classical neural networks. Motivated by this, we here for the first time prove a concise asymptotic limit for the loss functions of effectively all QNNs with approximately uniformly ran- dom initialization, and show that it unifies much of our previous understanding of QNN training behavior. We then use this new asymptotic description to prove a variety of novel results on the asymptotic training behavior of QNNs.
We achieve this by demonstrating that all QNNs have a natural algebraic structure described by Jordan subalgebra A of the complex Hermitian matrices H. (C) generated by {UoOU,}
under the anticommutator (A, B} = AB + BA. The Jordan subalgebras of HN (C) have been completely classified in the sense that they are expressible direct sum of subalgebras:
A
An
where for our purposes each Aa is isomorphic to the algebra of No x No Hermitian matrices H.a (Fa) over a field Fa that is one of the reals R, complex numbers C, or quaternions H. This allows us to show that the loss functions and first two derivatives of randomly initialized QNNs have random matrices over F.:
a concise asymptotic description in terms of Wishart-
W. = X o:
where Xa is No ra with i.i.d standard Gaussian entries over F& and Ta is called the degrees of freedom of W.. We give an explicit expression for Ta based on the structure of the network, and we find that it dictates the asymptotic behavior of the loss landscape. We call this connection between the algebraic structure of given QNN and its asymptotic Wishart process description Jordan
Girardi & Palma 2024 also study instances which do not suffer from harren plateaus, hut which only
achieve an asymptotical asymptoticalll-vaniihh vaniissiing improvement over random 0 in the optimization of Eq. !
2

---

Published as a conference paper at ICLR 2025
Barren Plateaus
Gaussian Processes (QNTK)
Reference
Corollary F
Reference
Corollary 5
McClean et al. (0188)
Garcia-Martin et al. (2023
F C, O Pauli
Cerezo et al.
2021
[F = C) a ei0 E Aut (A)
Girardi & Palmal (2024)
= C, O 1-local
Ragone et
(0024)
Garcla-Martin et al.
2024
R HH. O Pauli
Local Minima
Reference
Corollary 6
Anschuetz (2022)
F= {F = C)a < 2ra
Anschuetz & Kiani (2022)
You et al. (2022)
pDr
Table 1: Previous quantum neural network loss landscape results. We summarize (a represen- tative subset of) previous results in quantum neural network loss landscape theory, as described in Sec. I.I] We also state which limits of our Corollaries sults. Here, A is the Jordan algebra, F. are the fields, and Ta are the degrees of freedom parameters as defined in the associated corollary statements; O the objective observable; and p is the number of trained parameters.
and encompass these referenced re-
algebraic Wishart system (JAWS). We summarize in Table this JAWS description generalizes previous models of QNN loss landscapes which only held in specific parameter regimes.
The JAWS framework allows to prove many new, general properties of QNNs. First, we prove generalized barren plateau result for the variance of the loss function & (p; 0) of given QNN over its initialization:
Tr (() Tr ((pr)²
Vara (p:0)] = E
(4)
dimg (Aut (Aa))
where A 4 AA. is the Jordan algebra associated with the QNN, dimæ (Aut (Aa)) is the dimen- sion of the automorphism group of A. as a real manifold, and denotes projection into Aa (see Eq. 12p for mathematical definition). This extends previously known barren plateau results proved in the LASA framework (Fontana et al., 2024 Ragone et al. 2024 to the setting where neither nor O are elements of the algebra generating Aut (A,) ). also captures the gradient scaling in the
setting of so-called matchgate networks (Diaz et al. 2023), unifying barren plateau results beyond
just the LASA setting.
We are also able to calculate the asymptotic density of local minima at a loss value é = z for a QNN with associated Jordan algebra A θ.. Aa:
K (=) = * Jr (z; ka)
E., II I (ka.
daa Ki
exp
2a
(5)
ie., we show that it is a convolution of gamma distributions fr with shape and scale parameters ko and Sa, respectively, which we calculate Sec. The ka and s, are such that x (z) experiences a phase transition: local minima of network with parameters are concentrated near the global minimum if and only if the network overparameterized, which occurs when (up to a constant):
Tr (O")²
p Z max
(6)
Tr ((Om))
This result generalizes previous studies of local minima in QNNs (Anschuetz 2022 Anschuetz & Kiani 2022 to setting where the variational ansatz may have some sort of algebraic structure
Finally, we prove the necessary and sufficient conditions for a class of QNNs to asymptotically converge to a Gaussian process limit. Taken together, our results indicate that QNNs are efficiently trainable asymptotically using mrbbeem-independet optimization algorithms if and only if Eq. is not exponentially vanishing with the problem size and Eq. (6) is satisfied.
Anschuetz & Kiani (2022) consider local minima in local, shallow networks, where the Hilbert space can be decomposed into direct sum of Hilbert spaces associated with the light cones of local observables. Here, allow for general algebraic structures.
3

---

Published as a conference paper at ICLR 2025
2
PRELIMINARIES
2.1 QUANTUM NEURAL NETWORKS
We first review quantion neural networks (QNNs). These are defined by parameterized unitary:
Ug = 9p+1 πI expp(-i0,A.) 9s: t=p
often called an ansatz in the physics literature. Here, the A, are complex Hermitian N x N matrices and the gi are N x N unitary matrices. Generally, the gi and exp (-i0;A, may be constrained to belong path-connected Lie subgroup G of the unitary group U (N) to strengthen the inductive bias of the network (Meyer et al. (023). Such a constraint has also been used as theoretical model for shallow quantum networks, where G approximately the direct product Xa U (N.. of unitaries acting on local patches in the network (Anschuetz & Kiani 2022).
Training such networks involves minimizing an empirical risk of the form:
f (0)
Σ Ll (p:
Σ Irr (UopUjO). PER
R
R
Here, R can be thought of a data set comprising multiple input quantum states p-that is, p > E CNxN with trace I-and O CNNN is Hermitian. When O and the algebra elements generating Ug can be expressed as a sum of O (poly log (N)) Pauli matrices, such a loss can be estimated in O (poly log (N)) time a quantum computer (Nielsen & Chuang 2010b).
Our first main result is that all losses of the form of (p: 0) can be interpreted in terms of auto- morphisms of Jordan algebras. In preparation for discussing this connection, now review Jordan algebras.
2.2 JORDAN ALGEBRAS
A Jordan algebra over the reals is formally a real vector space V with commutative multiplication operation acting on u, E V satisfying the Jordan identity:
u ((u u) v) = (u u) (u
which ensures the associativity of the power. A simple example is the Jordan algebra H.V (C) of
N x complex-valued Hermitian matrices with o given by the anticommutator AoB AB BA.
The Jordan subalgebras A of HN (C) have been completely classified (Koecher 1999d) and are known to have a semisimple decomposition:
A
Ani
(10)
that is, is isomorphic to direct sum of subalgebras A.. For our purposes, each An isomor- phic to the algebra H. (F.) of N, x N₂ Hermitian matrices over field F,, that is one of the reals, complex numbers, or quaternions. We label these three cases with the integers 3g 1,2, 4, respectively, and call the representation of each of these algebras as Na No matrices over F., the defining representation. We call the A. the simple components of and say that A is simple when there is only a single Aa, i.e., when A _ HN (F) for some F.
The automorphism group Aut (.A) of such a Jordan algebra A is also known: its path- aath-connected components are isomorphic to direct products of the classical Lie groups SO (Na). SU (Na). and Sp (Na), each a subgroup of the corresponding Aut (An) (Koecher, 1999b, Orlitzky] 2024). As Lie group, each Aut (A.) has well-defined dimension as real manifold which denote as dimg (Aut (Aa))).
is also known that the Frobenius inner product Tr& (A'B) between A, B E Aa in any representa- tion R is always proportional to the Frobenius inner product in the defining representation (Koecher)
1999a). In particular, for any A, B E Aa C. A C H. (C), using Tra (.) to denote the trace in the defining representation of An and Tr to denote that of HN (C), we have:
Tr (AB) Tap Tra (AB)
(11)
4

---

Published as a conference paper at ICLR 2025
for some La > 0 that is independent of A, B. In what follows will continue to Tr (.) to denote the trace in the defining representation of H. (C) and Tra (.) to denote that A.. More details on Jordan algebras and some subtleties in their classification
provided in Appendix A.2
Finally, we define the projection of an algebra element A E A ① Aa into one the Aai to represent this, defining representation of A, and considering an orthonormal
the notation A Mathematically, letting Tra (-) denote the trace in the
3 basis (Ba,); ; of Aa S A, we have:
1 Σ Tra A) Ba.i-
(12)
We use the same notation for the equivalent projection of a Lie group element g E G X G, into one of the G.. Using (Bai); to denote an orthonormal basis of the Lie algebra generating Go, and using the observation that g = exp (oo, Ca,Bas)) for some {Ca, E [RJ,, we define:
g0= exp Σ CarbBaik
(13)
JORDAN ALGEBRAIC DESCRIPTIONS OF QUANTUM NEURAL NETWORKS
Recall the general form of QNN loss function:
8 (p: Tr (PU)OUA)
(14)
where
U, = 9p+1
II exp (-i0,A.,)) 9: i=p
(15)
generally may be such that the g. and exp (-i belong to some path-connected Lie subgroup G CN()N.
The UU generate under the anticommutator a Jordan subalgebra of the N x N Hermitian matrices HM (C). Due to this fact, as well as the classification results discussed Sec. 2.2 we may equivalently consider Eq. (14) in the following way:
O D. N (F.).
2. Ug E G X Ga. where each G. isomorphic to a subgroup of Aut (Aa) that is one SO (Na), SÚ (Na), Sp (Na).
3. For La the proportionality constant of Eq.
(p: 0) Σ la Tra (DU
(16)
We discuss the universality of this Jordan algebraic description in more detail in Appendix B.I
This description of QNN loss landscapes in terms of Jordan algebraic properties allows a simple way to classify various QNN architectures through the N., F., Ia. and Oa; we call this collection of algebraic objects Jordan algebraic Wishart system (JAWS). We will next tie this classification to the asymptotic properties of the loss landscape through the use of Wishart random matrices, justifying the use of "Wishart" in the nomenclature.
give
QUANTUM NEURAL NETWORKS ARE WISHART PROCESSES
Our main result is proving that the loss functions of wide QNNs is, as dim (A) form Wishart processes. Such processes can be written exactly in terms of the matrix elements of Wishart- positive semidefinite random matrices W. which are distributed as:
W
x
(17)
OOtthonormal with respect the Frobenius inner product in the defining representation of Aa.
5

---

certain additional assumptions on the Published as a conference paper at ICLR 2025
(4)
(8)
(c)
Cordhaned densty
Denal bscal mistr
Dereity
Mixed Input
Over parametteriio
Indergarrmetttiiien
Purn Input
Loss Suncionn value
vorivative value
Lass fundiion vaMu
Loss and derivative densities. (a) The loss density when the quantum neural network has pure input (i.e., rank-1) and when it has mixed input (i.e., of rank greater than 1). The dis- tributions are centered at the mean eigenvalue of the objective observable. The mixed input density also illustrates when the input is mixed when projected into any simple component of the Jordan al- gebra associated with the network. (b) The gradient density conditioned on a nonzero loss function value. The distribution is centered at zero. (c) The density of local minima when the quantum neu- ral network is radaraeeeeteri... overparameterized, and when some simple components of the associated Jordan algebra are underparameterrized and some overparameterized (mix of sectors).
Figure 1:
for X rectangular matrix with i.i.d standard normal entries over a field F (assumed if not otherwise stated). The number of columns of X is called the degrees of freedom of W in analogy with the degrees of freedom of x² di-ttrrbbeteed random variable; indeed, the diagonal entries of W (up constant scaling) i..d. x2 -d-distribtted with 3r degrees of freedom.
Our main result is as follows, stated informally here with formal statement deferred to Ap- pendix B.2
Theorem 1 (Quantum neural networks are Wishart processes, informal). Consider QNN with associated JAWS as in Sec. 3 initialized approximately uniformly at random. Let E be the optimum of the loss and o the mean eigenvalue of O Then, as dim (A) - co, there is convergence in joint distribution over at any 0:
e (p: ~s Σ Iaoa
Tra (p (p"Wa).
(18)
ra
The W. are each independent Wishart-distriuuted random matrices in the defining representation of Aa with
Tra (0
Ta
(19)
Tr (0#))²)
degrees of freedom, where [.] denotes rounding to the nearest integer.
An illustration of this distribution is provided in Figure the marginal distributions of matrix elements of the ansatz are approximately Gaussian distributed; this is made quantitative by bounding the errors in the associated joint characteristic functions over the inputs p. This is enough to demonstrate convergence of (p; 0) - l" to convolution of Wishart processes. We then prove a Welch-SSaterrtwwaite-ikk. result demonstrate an asymptotic conver- gence of this convolution to a Wishart process with degrees of freedom parameters given by Eq. (19). The full proof given in Appendix D We there also prove strengthened result- cottt--eneereeeee pointwise in the corresponding probability initialization of the network.
The proof follows by using the fact that
We not only prove the convergence of the QNN loss function & to that of Wishart process, but also its gradient, and Hessian jointly. In doing so we are able to show that the loss landscape of QNNs not just the landscape at single point parameter space, averaged over it it-vonverges to that of a Wishart process. We here only give an informal statement of the result in the special case when each p is rank-1, 1, but give the full distribution and formal statement in Appendix B.2 with proof Appendix [ This distribution when conditioned on a nonzero value for the loss is illustrated in Figure [b).

---

Published as a conference paper at ICLR 2025
Theorem 2 (Gradient distribution, informal). Consider the setting of Theorem 1 Assume each pa rank-1 in its defining representation. Let da be the standard deviation of the eigenvalues of O Then, as dim (A) + oc. the joint distribution of do,e (p: conditioned all (p": 00 asymptotically converges to the joint distribution of
2l.0 Tra (pº) 4=
Bazo 20 1a
V
Ga,iX..i.
(20)
Na
Here, the Xa are independent x-distributed random variables with max (2, degrees of freedom and the Ga are ii.dd standard normal random variables.
We also give an expression for the Hessian distribution at critical points. We informally report the result when each p" is rank-1 and the spectrum of each O is sufficiently concentrated around its mean; this latter condition typical of sums of low-weight fermionic (Feng et al. 2019) and local spin operators (Erdös & Schröder 2014). The full distribution and formal statement is once again given in Appendix B.2 with proof in Appendix [
Assume well that To + O for all a. Then, as dim (A) + co, the joint distribution of do Ro, E (p:0) (i 2 j) conditioned on all e ()";0) Za and all 0g. (p"((p":) = asymptotically converges to the joint distribution of
Theorem 3 (Hessian distribution, informal). Consider the setting of Theorem 2
lis = Σ 2Ta Tra (pa) Nă V 1ooa
Za
Ga,Ka.,Wa.tt
(21)
Here, the Xa. are independent x-distributed random variables with max 3a) degrees of free- dom and the Ga. are i.i.d. standard normal random variables. The W. are independent Wishart- distributed random matrices with BaTa degrees of freedom.
Both of these derivative results follow by carefully considering the joint distribution of elements of Wishart-distributed random matrix via the use of the Bartlett decomposition of Wishart-distriuuted matrices. Further simplifications follow by bounding the error in probability between various prod- uets of these marginal elements to simpler distributions.
NEW RESULTS IN LANDSCAPE THEORY FROM THE JAWS FRAMEWORK
With the full asymptotic distribution of QNN loss landscapes in hand, we are now able to prove several novel results QNN loss landscape theory.
5.1 BARREN PLATEAUS
The first implication of our results that will discuss is the unification of barren plateau sults (Fontana et al., 2024: Ragone et al. 2024) in the limit of large Jordan algebra dimension.
Corollary 4 (General expression for the loss function variance, informal). Let E be as in Theorem The variance of the loss over the initialization is:
Tr ((00))) Tr ((p7))
Vare [e (p:
= E
(22)
dimR (Aut (A.))
This follows immediately from the variance of elements of distart-distributee random matrices; see Appendix C.I for an explicit calculation. Despite the simplicity of the proof (given Theorem this result generalizes Theorem of Ragone et al. (2024). which only considered when either i0 or ip was in the algebra generating Aut (Aw). However, result holds for all variational ansatzes due to our general Jordan algebraic formulation of QNNs. Intriguingly, Tr ((a)²) can be thought of as probing the generalized entanglement (Barnum et al. 2004) of p with respect to the Jordan algebraic structure of AA. That is, entanglement induces barren plateaus, as previously seen in nonalgebraie setting by Ortiz Marrero et al. (202T). A similar phenomenon was also previously noted by Ragone et al. (024) with respect the Lie algebraic structure of LASAS.

---

Published as a conference paper at ICLR 2025
5.2
THE QUANTUM NEURAL TANGENT KERNEL
We now connect our results to the quantum neural tangent kemel (QNTK) literature (Lu et al.
2022
2023:
You et al., 2022, Gartia-Martin al. 2023: Girardi & Palma 2024 Gareía-
et al. 2024). The landmark result in this field is that, in certain settings, QNN loss functions are asymptotically Gaussian processes. However, this same body of work has noted that such Gaussian process description cannot generally hold. For instance, if the objective observable O is rank-1 1 and the algebra A associated with the QNN is the space of complex Hermitian matrices, is known that the loss is proportional to
proonentially-distrivvn
random variable (Boixo al. 2018).
Our results can be seen as a unifying model of neural network loss landscapes, including both when convergence to a Gaussian process is achieved and when it is not. Recall that Theorem demonstrated the asymptotic expression for the QNN loss at any 0 (left implicit) to be:
i (p) La Tra (p*Wa).
(23)
Ta
This correctly captures the exponential behavior when A the space of complex Hermitian matrices and rank (O) = = 1. This is because the diagonal entries of such a complex Wishart matrix are (up to constant) x disitrribt with two degrees of freedom each, which is identical to an exponential distribution.
Indeed, our more general result can be used to exactly characterize when the loss functions of QNNs asymptotically form Gaussian processes. First, we consider normalizing by some A such that N2 has nonvanishing variance asymptotically. Under this normalization, convergence to a Gaussian process occurs When.VV é has higher-order cumulants asymptotically vanishing. In other words, using K; (:) to denote the ith cumulant and R to denote the data set, this occurs when (see Appendix C.2 for an explicit calculation):
12 Tro ((09)3)
(Nē(p))
maa PER
(1).
(24)
PER 2
r2
We thus can state this pair of conditions as follows.
Corollary 5 (Exact conditions for convergence to Gaussian process, informal). Let & be as in
Theorem 7 Ne (p) is asymptotically Gaussian process over p R if and only Eq. (24) is satisfied and 22 Var [e (p)) (as in Corollary 4) is nonvanishing for all E R.
When these conditions met, the covariance is given by:
IC (p.p')
lim KN p') N
3a Tr ((09))
lim N2 N=200
Σ
I. (r (p" (Wa - -ra) Tra (p" (Wa - ra))]. 2 dimg (Aut (Aa))
(25)
It is now easy to why (for simple A) the 1 case does not converge Gaussian process: in this setting A must be o (N) such that Eq. (24) is satisfied and higher-order cumulants vanish, but for such choice the variance vanishing.
In Appendix C.2 we are also more explicit on the form of Eq. (25) We use known results for the mixed cumulants of Wishart matrix elements show that the covariance can be written as an explicit, quadratic expression in p and p² depending only on A. We further show in Appendix C.2 that, when the p E can be mutually diagonalized,
C.2
Tr (10))2
KN (p. 22 Σ dimg (Aut (A.))
Tr (p"p'O))
(26)
This form of the covariance can then be immediately fed into neural tangent kernel results to reason about the training behavior (Neal 1996: Choromanska al. 2015 Chaudhari 2018 Lee et al.,
8

---

Published as a conference paper at ICLR 2025
2018) and generalization ability (Jacot et al. 2018 Wei et al. 2022) of such networks. For instance, by the results of Girardi & Palmal (2024), this covariance suggests that gradient descent does not reach the global minimum in time polynomial in the model size for unstructured QNNs. This is discussed in more detail Appendix
5.3 LOCAL MINIMA
We end by examining the distribution of local minima of the loss landscape. This has been done previously in the more restricted setting where no structure is imposed on the ansatz or loss function outside of locality (Anschuetz) 2022] AAschhutz & Kiani, 2022). In this section we consider general QNN with p trained parameters under the conditions of Theorem 3 as well as assume that all
P
Yaa
(27)
BaTa
are held constant as we take the asymptotic limit dim (AA) + oc. We call the aa the overparame- terization ratios of a given QNN architecture, associated with the various simple components of the Jordan algebra A in the JAWS description of the network.
We calculate the distribution of local minima using the which gives the expected density of local minima of random field at a function value z. Anschuetz (2022) demonstrated that the assumptions for the Kac-Rice formula are satisfied for variational loss landscapes, and when rotationally invariant the p-torus the formula takes the form:
formula (Adler & Taylor 2007).
E CCroo (=)] (2xr) E [det (H.) 1 (H, = P) [G. P (e 2]
(28)
Here, [Crto (=)] is the expected density local minima function value 2, G2 is the gradient conditioned on L = and H, is the Hessian conditioned on L = and G = 0. In a slight abuse of notation, here P [-] denotes the probability density associated with the event
We can evaluate this expression using Theorems assume the addition of a regularization term of the form:
and 3 To simplify the expression here, we
R2 (0) = L 1 -
(29)
the loss; we also present here only the relative density rather than the total number of minima at loss function value z. The full expression counting local minima-both with and without the regularization of Eq. (29- are described detail in Appendix C.3
Corollary 6 (Density of local minima, informal). Consider the setting of Theorem 3 with an addi- tional regularization term of the form of Eq. (29). The density of local minima at a loss function value > 0 is, multiplicative leading order, given by the convolution over all a with Ya < 1:
* (r Tr
BaTa
(=)
(30)
2
"ara
ar
Here, fr (;k,0) denotes the gamma distribution with shape k and scale 0 parameters:
1 fo (1; r (k) BE
exp (-1)
(31)
Examples of this distribution plotted in Figure [T). where give finite width to the den- sity in the overparameterized regime-where all Ja 2 1-to represent potential finite-size effects. Intriguingly, the variance of this distribution when all Ta 1-the underparameterized regime- corresponds to the variance of the loss function itself, i.c., as given in Corollary 4 However, due the exponential tails of the gamma distribution, even when there no barren plateaus in the loss landscape there is only an exponentially small fraction of local minima in the vicinity of the global minimum in this regime. That is, both barren plateaus and poor local minima are potential obstructions to efficient trainability that must be taken into account in the design of practical QNNs.
6 CONCLUSION
Taken together, our results show that the natural model for quantum neural networks is not one of Gaussian processes obeying quantum neural tangent kernel, but rather of Wishart processes.

---

Published as a conference paper at ICLR 2025
This Wishart process model unifies all of the recent major thrusts in calculating properties of quan- tum neural network loss landscapes. Indeed, our results allow us to propose a simple operational definition for the "trainabilitty of quantum neural networks, which to date has been term used heuristically without any formal definition [
Definition (Trainability of quantum neural networks). Consider QNN with p trained parameters composed of N x N unitary matrices. Let A = θ. Aa be the corresponding Jordan algebra as in Sec. Define the degrees of freedom parameters:
Tra (Om)2
Ta
Tra (00))²)
(32)
We say that the QNN is trainable if and only if, as N + co, the QNN satisfies:
1. Absence of barren plateaus (Corollary 4):
Tr (09)² Tr ((0)))
Ω (poly (log (AV))
(33)
dimg (Aut (Aa))
2. Absence of poor local minima (Corollary 6):
p 2 max Bata.
(34)
Given knowledge of A, this yields quantum algorithm for determining the asymptotic trainability of a QNN architecture: one need only measure Tr (Om))) and Tr ((p"))) on a quantum com- puter. This can be done through the measurement of basis elements of A given copies of p and, for the former, block encodings of O as prepared by the standard linear combination of unitaries sub- routine (Childs & Wiebe 2012). ). We provide explicit error bounds for this procedure in Appendix
Practically, where does this leave quantum neural networks? For one, it seems unlikely that there exists any computational quantum advantage during the training of QNNs outside of HHL-like speedups (Harrow al., 2009 Biamonte al. (017)) and training algorithms that leverage some existing knowledge of the data to be learned (Liu et al.] Hastings & O' O'Donnell [2022] Huang let al., 2024). This is due to efficient classical simulation algorithms for quantum systems alge- braically constrained to explore a low-dimensional subspace (Anschuetz et al. 2023a Goh et al.,
2023), y as is required for efficient trainability per Definition 7 This was noted in Cerezo et al. (2023) for deep QNNs, where the authors postulated that deep QNNs exhibiting no barren plateaus are clas- sically simulable (outside of certain special cases). Our results demonstrate similar phenomenon for shallow QNNs: poor local minima in polynomially-sizen circuits can only be avoided when the effective Hilbert space dimension grows at most polynomially quickly with the system size. This leaves the space for a practical, superpolynomial quantum advantage when training with a problem- agnostic algorithm such as gradient descent even narrower than previously believed: a veritable Amity Island in a sea of negative results. One ray of hope is the known existence of polynomial quantum advantages during inference in such a setting (Anschuetz et al.) 2023b] Anschuetz & Gao, 2024]. Intelligent warm-starting of the optimization procedure may be another way to circumvent poor training (Puig-i-Valle et al. 2024), though more must be done to fully understand how pre- cise warm-starting must be for training to be efficient. Given the exact asymptotic form of the loss landscape we give here, such an analysis may be possible in the future.
Our work gives unified understanding of quantum neural networks as Wishart processes. Great strides have been made in the classical machine learning literature in understanding the training dynamics (Choromanska et al. 2015 Chaudhari] 2018, Jacot et al.) 2018 Allen-Zhu et al.) 2019) and generalization behavior (Jacot et al. 20188) Wei et al. 2022) of classical neural networks via their connections to Gaussian processes, which unfortunately only port over in the specific settings where the Wishart process itself approaches Gaussian process. Our results encourage an understanding of how specific properties of Wishart processes, not just Gaussian processes, influence the learning behavior of quantum networks in order to more fully grasp how quantum neural networks learn.
'The concept of "efficient learning" has been previously studied GGil-Fuster et al. 2024), but give the first operational definition in terms of the structural properties of a given QNN architecture.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
The main results discussed in Secs. and are formally stated- an explicit listing of formal assumptions-in Appendix Definitions and background necessary for these formal theorem and assumption statements are laid in Appendix A Formal proofs of the main results are given in Appendix D Formal statements and proofs of the corollaries discussed in Sec. [ are given in Appendix C proof of the claim made in Sec. [ that one can efficiently estimate the quantities in Definition 7 using an LCCU block encoding is given in Appendix E A discussion of the settings
where our results do not hold due to a violation of our formal assumptions is given in Appendix
The remainder of the Appendices prove claims and helper lemmas used in Appendices
B C and
ACKNOWLEDGMENTS
E.R.A. is grateful to Pablo Bermejo, Marco Cerezo, Diego Gareía-Martin, Bobak T. Kiani, Martín Larocca, Thomas Schuster, and Alissa Wilms for enlightening discussion and suggestions that aided in the preparation of this manuscript. E.R.A. also thanks Nathan Wiebe for advocating for formal definition of trainability in quantum neural networks the PennyLane Research Retreat of 2023, which inspired parts of this work. E.R.A. was funded in part by the Walter Burke Institute for Theoretical Physics Caltech.
REFERENCES
Robert J. Adler and Jonathan E. Taylor. Random Fields on Manifolds, pp. 301-330. Springer New York, New York, USA, 2007. ISBN 978-0-387-4811644 doi: 10.1007/978-0-3887-11 1818116661122
Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-parameteriiztion. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Ma- chine Learning Research, pp. 242-252, Long Beach, USA, 6 2019. PMLR. URL https: 7 p/prceediiggs m www.presss/ee v97/a1 len- zhul 9a. html
Eric R. Anschuetz and Xun Gao. Arbitrary polynomial separations in trainable quantum machine learning. CoRR, abs/2402.08606, 2024. doi: 10.48550/ARXV22402..085
Eric R. Anschuetz and Bobak T. Kiani. Quantum variational algorithms are swamped with traps.
Nat. Commun., 13:7760, 2022. doi: 10. 10.70388/41446--02---33
Eric R. Anschuetz, Andreas Bauer, Bobak T. Kiani, and Seth Lloyd. Efficient classical algorithms for simulating symmetric quantum systems. Quantum, 7:1 189, 11 2023a. ISSN 2521-327X. doi:
10.22331/4- 11-28-1--2--111
Eric R. Anschuetz, Hong- Ye Hu, Jin-Long Huang, and Xun Gao. Interpretable quantum advantage in neural sequence learning. PRX Quantum, 4:020338, 2023b. doi: 10.1 10111RXXuuuuuuum... 020338.
Eric Ricardo Anschuetz. Critical points in quantum generative models. In Katja Hofmann, Alexan- der Rush, Yan Liu, Chelsea Finn, Yejin Choi, and Mare Deisenroth (cds.), The Tenth International Conference Learning Representations. OpenReview, 2022. URL httph:tt:///t.. httpps://oopnrevieett
neet/ooru?1dd=211
Howard Barnum, Emanuel Knill, Gerardo Ortiz, Rolando Somma, and Lorenza Viola. A subsystem- independent generalization of entanglement. Phys. Rev. Lett., 92:107902 Mar 2004. doi: 10.
1103/PhysRevLett92 107902.
Istvan Berkes and Walter Philipp. Approximation thorems for independent and weakly dependent random vectors. Ann. Probab., 7(1):29-54, 1979. doi: 10.1214/aop/117666 176995146.
Jacob Biamonte, Peter Wittek, Nicola Pancotti, Patrick Rebentrost, Nathan Wiebe, and Seth Lloyd. Quantum machine learning, Nature, 549((671): 2017. doi: 10. 11.108//auuur2474.
Sergio Boixo, Sergei V Isakov, Vadim N Smelyanskiy, Ryan Babbush, Nan Ding, Zhang Jiang, Michael Bremner, John M Martinis, and Hartmut Neven. Characterizing quantum supremacy in near-term devices. Nat. Phys., 14(6):595-60) 2018. doi: 10.1038/441567-018-00

---

Published as a conference paper at ICLR 2025
Dennis D. Boos. A converse to Scheffe's theorem. Ann. Stat., 13(1):423 427, 1985. doi: 10.1214/ aos/1176346604.
Marco Cerezo, Akira Sone, Tyler Volkoff, Lukasz Cincio, and Patrick J Coles, Cost function depen- dent barren plateaus shallow parametrized quantum circuits. Nat. Commun., 12(1):1791-1002 2021. doi: 10. 14tp///4466-0211-111
Marco Cerezo, Martin Larocca, Diego Garcia-Martín, Nelson L. Diaz, Paolo Braccia, Enrico Fontana, Manuel S. Rudolph, Pablo Bermejo, Aroosa Ijaz, Supanut Thanasilp, Eric R. Anschuetz, and Zoë Holmes. Does provable absence of barren plateaus imply classical simulability? Or, why we need to rethink variational quantum computing. CoRR, abs/2312.09121, 2023. doi: 10.4850/ARXIV2312.0011
Shantanav Chakraborty. quantum computers. q-2024- 1144-10-10 10-16.1496.
Implementing any linear combination of unitaries on intermediate- Quantum, 8: 8:1496, October 2024. ISSN 2521-327X. doi: 10.22331/
Pratik Anil Chaudhari. A Picture of the Energy Landscape of Deep Neural Nenworks. PhD the- sis. University of California, Los Angeles, 2018. UtL/:tttees//eecoolrrrepp....//p
item/26h57877
Chi-Fang Chen, Alexander M. Dalzell, Mario Berta, Fernando G. S. L. Brandão, and Joel A. Tropp- Sparse random Hamiltonians are quantumly easy. Phys. Rev. X, 14:01 2024. doi: 10.1 103/ PhysRevX.
Andrew M. Childs and Nathan Wiebe. Hamiltonian simulation using linear combinations of unitary operations. Quantum Info. Comput., 12(11-12):011-22,, November 2012. ISSN 1533-7146. doi:
10.26421/1C12211-111-11
Anna Choromanska, Mikael Henaff, Michael Mathieu, Gerard Ben Arous, and Yann LeCun. The loss surfaces of multilayer networks. In Guy Lebanon and S. V. N. Vishwanathan (eds.), Pro- ceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, vol- ume 38 of Proceedings of Machine Learning Research, pp. 192-204, San Diego USA, 2015. m.r. press http:////ccooooaasa....c
PMLR.
Benoît Collins, James A Mingo, Piotr Sniady, and Roland Speicher. Second order freeness and fluctuations of random matrices. III: Higher order freeness and free cumulants. Doc. Math., 12: 1-70, 2007. doi: 10.4171/DM/220.
Amir Dembo and Ofer Zeitouni. General Principles, pp. 115-174. Springer Berlin Hei- delberg, Berlin, Heidelberg, Germany, 2010. ISBN 978-3-644-0931117. doi: 10.1007/
978-3-642-03311-22
N. Diaz, Diego Garefa-Martie Sujay Kazi, Martin Larocca, and M. Cerezo. Showcasing barren plateau theory beyond the dynamical Lie algebra. arXiv,, abs/2310.1 1505, 2023. doi: 10.48550/ arXiv.2310.1 11505.
Alexander Dubbs, Alan Edelman, Plamen Koev, and Praveen Venkataramana. The Stia-Wishart ensemble. J. Math. Phys., 5488)00835 08 2013. ISSN 0022-2488. doi: 10. 4818304.
László Erdōs and Dominik Schröder. Phase transition in the density of states of quantum spin glasses. Math. Phys. Anal. Geom., 17(3).441-666 2014. doi: 10. 10477/010000011111444
Renjie Feng, Gang Tian, and Dongyi Wei. Spectrum of SYK model. Peking Math. J., 2(1):41-70, 2019. doi: (10077/42543-018-00000
Enrico Fontana, Dylan Herman, Shouvanik Chakrabarti, Niraj Kumar, Romina Yalovetzky. Jamie Heredge, Shree Hari Sureshbabu, and Marco Pistoia. Characterizing barren plateaus quan- tum ansätze with the adjoint representation. Nat. Commun., 15(1):7171, 2024. doi: 10. 1038/ 941467-002-499100w
Diego Martin,Martin, Martin Larocca, and Marco Cerezo. Deep quantum neural networks form Gaussian processes. CoRR, abs/2305.09957, 2023. doi: 10.48500/AXXI12205.00955
12

---

Published as a conference paper at ICLR 2025
Diego García- Garcfa-Martín, Paolo Braccia, and Marco Cerezo. Architectures and random properties of symplectic quantum circuits. CoRR, abs/2405. 10264, 2024. doi: 10.48550/ARXIV.2405.. 10264.
Elies Gil-Fuster, Casper Gyurik, Adrián Pérez-Salinas, and Vedran Dunjko. On the relation between trainability and dequantization of variational quantum learning models. CoRR, abs/2406.07072,
2024. doi: 10:4850/ARXXV.2406.0000
Filippo Girardi and Giacomo De Palma. Trained quantum neural networks are Gaussian processes. CoRR, abs/2402.08726, 2024. doi: 10.48550/ARXIV.2002. 20..276...
Matthew L.. Goh, Martin Larocca, Lukasz Cincio, Cerezo, and Frédéric Sauvage. Lie-algebraic classical simulations for variational quantum computing. arXiv, abs/2308.01432, 2023. doi:
10.48550/axXi.2208.014222
Alfred Haar. Der Masshegriff in der Theorie der kontinuierlichen Gruppen. Ann. Math., 34(1): 147-169, 1933. ISSN 0003486X. doi: 10.2307/1968344
Aram W Harrow and Saced Mehraban. Approximate unitary I-designs by short random quantum circuits using neirest- and long-range gates. Commun. Math. Phys., 401(2):151-1166, 2023. doi: 10.1007/00022--22--4400
Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. Quantum algorithm for linear systems of equations. Phys. Rev. Lett. 103:150502, 10 2009. doi: 10. 100
Matthew B. Hastings and Ryan O'Donnell. Optimizing strongly interacting fermionic Hamilto- nians. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2022, pp. 776-789, New York, USA, 2022. Association for Computing Machinery. ISBN 9781450392648. doi: 10. 145/3519935.3519996
Fumio Hiai and Dénes Petz. Eigenvalue density of the Wishart matrix and large deviations. Infin. Dimens. Anal. Quantum Probab. Relat. Top., 01(04))633-666, 1998. doi: 10.1142/
S021900279900033X..
Brice Huang and Mark Sellke. theeshol for multi-species spherical spin glasses. CoRR, abs/2303.12172, 2023. doi: 1044850//AXXV.2333.
Hsin-Yuan Huang, Yunchao Liu, Michael Broughton, Isaac Kim, Anurag Anshu, Zeph Landau, and Jarrod R. McClean. Learning shallow quantum circuits. In Proceedings of the 56th Anmual ACM Symposium on Theory of Computing, STOC 2024, pp. 1343-1351, New York, USA, 2024. Association for Computing Machinery. ISBN 9798400703836. doi: 10.1145/6118200.3649979
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: convergence and gen- cralization in neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 8580 8589, Red Hook. USA, 2018. Curran Asso- /dl acm. org. do abs 10. 5555 / 3377757 3327948
ciates Inc.
Tiefeng Jiang. The entries of Haar-invariant matrices from the classical compact groups. J. Theor. Probab., 23 2010. doi: 100000009099500-0022220
Eytan Katzav and Isaac Pérez Castillo. Large deviations of the smallest eigenvalue of the Wishart- Laguerre ensemble. Phys. Rev. E, 82:040104, 10 2010. doi: 10. 10.100//hhyRReeE..2..0000
A.I. Khuri, T. Mathew, and D.G. Nel. A test determine closeness of multivariate Satterthwaite's approximation. J. Multivar. Anal., 51(1):201-209 1994. ISSN 0047-259X. 1994.1057.
doi: 10.1006/jmm
Max Koecher. Jordan algebras, pp. 53-72. Springer Berlin Heidelberg, Berlin, Heidelberg, Ger- many, 1999a. ISBN 97833-540-48402-88 doi: 10.10077BFF00096288,
Max Koecher. Real and complex Jordan algebras. pp. 73-92. Springer Berlin Heidelberg, Berlin, Heidelberg, Germany, 1999b. ISBN 978-3-540-44840-88 doi: 10.1007/B600966889
Max Koecher. Jordan algebras and omega domains, pp. 126-126. Springer Berlin Heidelberg,
Berlin, Heidelherg, Germany, 1999c. ISBN 978-3-540-48402-8 doi: 10.1007/BFM009629..
13

---

Published as a conference paper at ICLR 2025
Max Koecher. Half-spaces, pp. 127-155. Springer Berlin Heidelberg, Berlin, Heidelberg, Germany, 1999d. ISBN 978-3-540-48402-8 doi: 10.1007/BF60096292.
Martin Larocca, Piotr Czarnik, Kunal Sharma, Gopikrishnan Muraleedharan, Patrick Coles, and M. Cerezo. Diagnosing barren plateaus with tools from quantum optimal control. Quantum, 6: 824, 2022. ISSN 2521-327X.
doi: 10.22331/4-2222-09--944444
B. Laurent and P. Massart. Adaptive estimation quadratic functional by model selection. Ann. Stat., 28(5):1302-13388 2000. doi: 01/14/////11155533999
Jaehoon Lee, Yasaman Bahri, Roman Novak, Sam Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as Gaussian processes. In Yoshua Bengio and Yann Le- Cun (eds.), The Sixth International Conference on Learning Representations, Vancouver, Canada, 2018. OpenReview.
htt.htttp:///opeeeee net pt pet.:/pp
Andy C. Y. Li and Imanol Hemmandez. Noise-induced transition in optimal solutions of variational quantum algorithms. arXiv, abs/2403.02762, 2024. doi: 10.48850//rXiv.2440..2222
Fei Li and Yifeng Xue. The density functions of the singular quaternion normal matrix and the singular quaternion Wishart matrix. Commun. Stat. Theory Methods, 39(18):3316-3311 2010.
doi: 10. 101100000000900000000 0920903259666.
Junyu Liu, Francesco Tacchino, Jennifer R. Glick, Liang Jiang, and Antonio Mezzacapo. Repre- sentation learning via quantum neural tangent kernels. PRX Quantum, 3:030323, 2022. doi: 10. 10.110//PXXQuantum..38833 3.00030.
Junyu Liu, Khadijeh Najafi, Kunal Sharma, Francesco Tacchino, Liang Jiang, and Antonio Mezza- capo. Analytic theory for the dynamics of wide quantum neural networks. Phys. Rev. Lett., 130: 150601, 2023. doi: 10. 103/PhysRevl.ett
150601.
Yunchao Liu, Srinivasan Arunachalam, and Kristan Temme. A rigorous and robust quantum speed-up in supervised machine learning. Nat. Phys., 17(9):1133-101, 2021. doi: 10.
s41567-021-012277.2
A Marčenko and L Pastur. Distribution of eigenvalues for some sets random matrices. Math. USSR Sb., 1(4):457, 4 1967. doi: 10. 10.007051166770001m h0A.0000000000000000000000000000
Jarrod R McClean, Sergio Boixo, Vadim N Smelyanskiy, Ryan Babbush, and Hartmut Neven. Bar- ren plateaus in quantum neural network training landscapes. Nat. Commun., 9(I):4812, 2018. doi: 10. 147-1414--7099999
Elizabeth S. Meckes. Concentration of Measure, pp. 131-160. Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, UK, 2019. doi:
Antonio Anna Mele, Armando Angrisani, Soumik Ghosh, Sumeet Khatri, Jens Eisert, Daniel Stilek França, and Yihui Quek. Noise-induced shallow circuits and absence of barren plateaus. arXiv, abs/2403. 13927, 2024. doi: 10.48550//rxiv.243..1
Johannes Jakob Meyer, Marian Mularski, Elies Gil-Fuster, Antonio Anna Mele, Francesco Arzani, Alissa Wilms, and Jens Eisert. Exploiting symmetry in variational quantum machine learning. PRX Quantum, 4:010328, 2023. doi: 10. 1003/PRXQuantmm 4.00000.
James A. Mingo and Mihai Popa. Real second order freeness and Haar orthogonal matrices. J. Math. Phys., 54(5):051701, 5 2013. ISSN 0022-2488. doi: 10. 488446//..8804668
James A. Mingo and Roland Speicher. Second order freeness and fluctuations of random matrices: Gaussian and Wishart matrices and cyclic Fock spaces. J. Funct. Anal., 235(1):226-270, 2006.
ISSN 0022-1236. doi: 10. 10.1016/j.ifa.2000 10.007.
Radford M. Neal. Priors for Infinite Networks, pp. 29-53. Springer New York, New York, USA, 1996. ISBN 978 118-1-4612-0745-0 doi: 10.1007/978-1-440 4-61----5--------000-
Michael A. Nielsen and Isaac L. Chuang. The quantum Fourier transform and its applica- tions, pp. 216-247. Cambridge University Press, Cambridge, UK 2010a. doi: 10.1017/ CBO978051 1976667779976667000
14

---

Published as a conference paper at ICLR 2025
Michael A. Nielsen and Isaac L. Chuang. The quantum Fourier transform and its applica- tions, pp. 216-247. Cambridge University Press, Cambridge, UK, 2010b. doi: 10.1017/ #B09780511976667.o
Michael Orlitzky. Jordan automorphisms and derivatives of symmetric cones. Linear Algebra Its Appl., 2024. ISSN 0024-3795. doi: 10. laa1//..aa.2224.04
Carlos Ortiz Marrero, Mária Kieferová, and Nathan Wiebe. Entanglementtinduced barren plateaus. PRX Quantum, 2:040316, Oct 2021. doi: (0.L103/PRXQuaatumm22.00000
Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Alán Aspuru-Guzik, and Jeremy L. O'Brien. A variational eigenvalue solver on photonic quan- tum processor. Nat. Commun., 5(1):4213, 2014. doi: 10. 10.0038/ccmmm5213.
Ricard Puig-i-Valls, Marc Drudis, Supanut Thanasilp. and 7oë Holmes. Variational quantum simulation: a case study for understanding warm starts. CoRR, abs/2404.10044, 2024. doi: 10.48550/ARXIV.2444..0
10044.
Michael Ragone, Bojko N Bakalov, Frédéric Sauvage, Alexander F Kemper, Carlos Ortiz Marrero, Martín Larocca, and M Cerezo. A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits. Nat. Commun., 15(1):7172, 2024. doi: 10. 10.1038//41446--044449924
C. E. Redelmeier. Quaternionic second-order freeness and the fluctuations large symplectically invariant random matrices. Random Matrices: Theory Appl., 10(02):2150017, 2021. doi: 10.
11425520103263215001799
C. Emily I. Redelmeier. Genus expansion for real Wishart matrices. J. Theor. Probab, 24(4):1044- 1062, 2011. 1. doi: 10.1007/s1095--10000777
Catherine Emily Iska Redelmeier. Real second-order freeness and the asymptotic real second-order freeness of several real matrix models. Int. Math. Res. Not., 2014(22)33333-333,, 3 2012. ISSN 1073-7928. doi: 10.10933/mm//m1443.
Alain Rouault. Asymptotic behavior of random determinants in the Laguerre, Gram and Jacobi ensembles. ALEA Lat. Am. J. Probab. Math. Stat., 3:181-230, 2007. URL /https://hal.
science 00088079
F. E. Satterthwaite. An approximate distribution of estimates of variance components. Biometrics Bull., 10-111 1946. ISSN 00994987. doi: 10.2307/3002019.
Henry Scheffé. A useful convergence theorem for probability distributions. Ann. Math. Stat., 18(3):
434-438, 1947. doi: 00.1214/aoms// / 77730390.
M.S. Srivastava. Singular Wishart and multivariate beta distributions. Ann. Stat., 31(5):1537-1566) 2003. doi: 00114/////16670000
V. Strassen. The existence probability measures with given marginals. Ann. Math. Stat., 36(2):
423-439. 1965. doi: 10. 10.1221/aommt 177700001..
Eliran Subag. The complexity of spherical p-spin models-a second moment approach. Ann. Probab., 45(5))3385 3450, 2017. doi: 10.1214/16-AOP/
Roman Vershynin. Concentration of Sums of Independent Random Variables, pp. 11-37. Cambridge Series Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, UK, 2018. doi: 10. 10823 0108231159.0000 505.
Samson Wang, Enrico Fontana, Marco Cerezo, Kunal Sharma, Akira Sone, Lukasz Cincio, and Patrick J Coles. coise-induced barren plateaus in variational quantum algorithms. Nat. Commun., 12(1):6961, 2021. doi: 1031038/ 1e..0088/414477-2111 467-021-220444
Alexander Wei, Wei Hu, and Jacob Steinhardt. More than a toy: Random matrix models pre- dict how real-world neural representations generalize. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th Inter- national Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 23549-23555 PMLR, 7 2022. URL ht tatpp:: http::///oocccedeeeeeoe .mrr, www.mresss////pop... v1 62, wei 22a html
15

---

Published as a conference paper at ICLR 2025
B. L. Welch. The generalization of 'Student's' problem when several different population variances are involved. Biometrika, 34(1-2):28-33 35, 01 1947. ISSN 0006-3444. doi: 10.1093/biomet/34. 1-2.28.
Xuchen You, Shouvanik Chakrabarti, and Xiaodi Wu. A convergence theory for paea-eerameterizedd variational quantum eigensolvers. CoRR, abs/2205. 12481, 2022. doi: 10.48550/ARXIV.2205.. 12481.
Soonyu Yu, Jaena Ryu, and Kyoohong Park. A derivation of anti- Wishart distribution. J. Multivar. Anal., 131:121-125, 2014. ISSN 0047-259X. doi: 10.10166f.jmva22114.0.000
A PRELIMINARIES FOR FORMAL DISCUSSION OF RESULTS
We begin by reviewing concepts that will use in proving our results. We also give summary of the notation we use throughout in Table 2
A. QUANTUM NEURAL NETWORKS
We first review quantum neural networks (QNNs). These are defined by a parameterized ansatz
U (0)
II exp (-i8,A,),
(35)
which belongs to some path-connected subgroup of the unitary group U (N). Though U (0) can in principle parameterize all of U (N), it is often taken to instead parameterize some path-connected subgroup S (N). This might due to enforcing some global structure on the model (Myyer al. 2023). can be model of the finiteness of the reverse light cones of shallow quantum
circuits (Anschuetz Kiani 2022).
Given choice of ansatz, the goal of QNN is to minimize an empirical risk of the form:
(0)
Ee(0:p)
((0
ET PER
PU PU (U) 0).
(36)
R
R
PER
Here, can be thought of data set comprising multiple input states p. and the loss function. His- torically, when [R] = QNNs have been referred to variational quanium algorithms (VQAs) (Pe- ruzzo al. 2014) due to their connection to finding variational approximations the ground states of quantum Hamiltonians. There are known quantum-classical separations for the expressivity of quantum neural networks even when taking into account the requirement that the training procedure is efficient (Liu et al. 2021: 1: Hastings & O' Donnell] 2022 Huang et al. 2024), though they require very specific training algorithms that take advantage of the structure of the data.
There has been recent hope that, with enough ansatz structure, efficient training may follow just via a simple application of gradient descent. This follows from the Lie algebra-sup eieebra-uupported ansatz (LASA) literature, where it has been shown that if the generators iA, of the ansatz. U well the (scaled) objective observable iO belong to the same Lie algebra g g-called the dynamical Lie alge- bra (Larocca et al.) (0022) generating G, gradients scale inversely with the dimension of g (Fontanal ct al. 2024 Ragone et al., 2024). It is also conjectured that when dim (g) scales polynomially with the system size there exist polynomial-depth ansatzes that do not have poor local minima, which together with the large gradients would imply efficient trainability of these loss functions via gra- dient descent. Conditioned on this conjecture there have been results demonstrating expressivity separations in quantum machine learning where the QNN is efficiently trainable through a simple application of gradient descent (Anschuetz al, 2023b Anschuetz Gao 2024).
Though the LASA framework gives sufficient conditions for loss functions to have large gradients, is known that they are not necessary. One example of this was demonstrated in Diaz et al. (2023), where it was shown that parameterized matchgate circuits with objective observable given by constant-degres polynomials in Majorana fermions are efficiently trainable though they are not a part of the LASA setting. We claim that both settings are special cases of Jordan algebraic under- standing of variational loss functions. In preparation of discussing this connection we now review Jordan algebras.
16

---

Published as a conference paper at ICLR 2025
No
Natural numbers including 0
N1
Natural numbers excluding 0
[x]
Natural numbers from 1 through n
H. (F)
Jordan algebra of n Hermitian matrices over F
IF
Field, here one of R, C, HI
B
3 = 1, 2, 4 when associated field IF = IR, C, respectively Dimension of as a vector space over I
dimy (-)
No (0,
Standard normal distribution over F (given by ())
B- Wishart distribution of n X n matrices with r degrees of freedom and scale matrix Σ Convergence in distribution
W (r, 2
2
Convergence in probability
Hadamard product
Aa
Simple components of semisimple Jordan algebra A
Go
Lie group isomorphic to a connected component of Aut (A.)
ga
Lie algebra generating G..
Defining representation of the projection of into Aa Dimension of vector space on which the defining representation of A acts
Na
N
Sum of N₂.
Tra (-)
Trace of - in the defining representation of An
(r)-)
Trace of according to its representation in H (C) 2 A
-llop
Operator norm of E A,, in its defining representation
Trace (nuclear) norm of E Ac in its defining representation
Frobenius norm of E A., in its defining representation
O (poly ()))
(N*) for some constant >> 0
WLOG
without loss of generality with high probability
w.h.p.
Table 2: Table of notation. Notation used in the presentation of our results are given in the left column with corresponding meaning given in the right column. We also note here that exponents written as decimals throughout our results are chosen somewhat arbitrarily, i.e., can be improved by any sufficiently small constant > 0 in the exponent. Finally, we note that we will often forgo writing ". "...a a sequence of [ansatzes, loss functions, observables, QNNs, ...]..." when describing our asymptotic convergence results for brevity.
A.2 JORDAN ALGEBRAS
A Jordan algebra over the reals is formally a real vector space V with a commutative multiplication operation acting on u, E V satisfying the Jordan identity:
u ((u o u) o v) = (uo o (u
(37)
which ensures the associativity of the power. A simple example of Jordan algebra over the reals is the real algebra H. (C) of N x N complex-valued Hermitian matrices with o given by half the anticommutator. In particular, for any finite-dimensional quantum system both O and the inputs p in Eq. (36) belong to HN (C). We emphasize that though this algebra is typically written in terms of complex matrices is still real Jordan algebra. This is because, for instance, i & R multiplying a Hermitian matrix is no longer Hermitian. In fact, the Jordan algebra of Hermitian matrices is Euclidean (or formally real) as it satisfies the defining property (Koecher) 1999ch:
AA, B E A, A o A + B B = 0 + A= B
(38)
is apparent that all subalgebras of a formally real algebra are also formally real.
The real vector space V with which a Jordan algebra is associated also has a natural linear transfor- mation L (w):
v, which can be viewed as the Jordan algebraic analogue of the adjoint representation of Lie algebras. This linear transformation gives rise to the canonical trace form:
L (w)
(39)
(u, r) Tr (T (u v)).
(40)
17

---

Published as a conference paper at ICLR 2025
When is nonsingular call the associated Jordan algebra A semisimple. As an example, for the algebra 7 (IF) of Hermitian matrices over the field F, T (u, v) is just the Frobenius inner product between u and v. Just as is the case for semisimple Lie algebras, all symmetric bilinear forms a semisimple Jordan algebra are identical up to an overall scaling. Specifically, for any symmet- ric bilinear form Ø, there exists element 2 in the center of A such that (Theorem 10, KKochhr 1999a)):
a (u, v) = (z u, v)
(41)
For A = H (C) in the defining representation the center is just real multiples of the identity matrix, i... there exists a real I 2 0 Ouch that:
Io (u, (u, v).
(42)
We call I the index of a in analogy with Fontana et al. (2024) for Lie algebras.
Though perhaps not as famous as the classification of compact Lie groups, the semisimple Euclidean Jordan algebras have also been classified.
Theorem 8 (Classification of semisimple Euclidean Jordan algebras (Koecher 1999d)). Any semisimple Euclidean Jordan algebra is isomorphic to direct sum of the simple Euclidean Jor- dan algebras:
CN for N 2 the spin factor, with vector space equal IR and given by the operation
o y = (x: y, 31 y1 where denotes with the first coordinate projected out;
HN (R) N 2 3: N x N symmetric matrices over R, with o half the anticommutator;
* HN (C) for N 2 3: N X N Hermitian matrices over C, with o half the anticommutator;
HN (TH) for N 2 3: N X N Hermitian matrices over H. with half the anticommutator:
+H3 (O): 3 Hermitian matrices over O, with half the anticommutator.
We use the term defining representation to speak of the described matrix representations of these algebras. As the Hermitian octonion case is not an infinite family it is often called exceptional. We here are interested in asymptotic sequences of Jordan algebras we will not be considering the exceptional case.
Every Jordan algebra A has associated with it an automorphism group Aut (A). As an example, for H. (R) in the defining representation this is just given by the action of conjugation (under the usual matrix multiplication) by orthogonal matrices. More generally, the nonexceptional simple cases have automorphism groups (Orlitzky 2024):
Aut (Cx): left-action of (1} (NN-)
Aut (HL (R)): conjugation action PO (N);
Aut (H (C)): disjoint union of the conjugation action of PU (N), and the transpose
followed by the conjugation action of PU (N);
Aut (H (HI)): conjugation action of PSp (N).
We are here primarily interested in the connected component Aut, (-) S. Aut (-) containing the identity transformation, we will only concern ourselves with the path-connected automorphism subgroups:
SO (N 1) z Aut, (Cx) C Aut (CN);
SO (N) = Aut NH (RR) C Aut (H (IR));
SU (N) Aut, (H (C)) Aut (H (C));
SNN = Aut, (H (IH)) Aut NH (HH)).
18

---

Published as a conference paper at ICLR 2025
Surprisingly, the automorphism groups of semisimple Jordan algebras also classified. Given decomposition semisimple Euclidean Jordan algebra A into simple components:
Aa
(44)
the connected component of Aut, (AA) C Aut (A) containing the identity is isomorphic to the direct product (Theorem 10, Koecher 1999b)):
Aut (A) X Aut (Aa).
(45)
A.3 C-APPROXIMATE t-DESIGNS OVER Aut, (AA
In order to discuss the structure of the loss function in any detail we will need to consider some choice of randomness over loss functions. achieve this we will use c-approximate t-designs over Aut, (A). Our use of these designs formalizes the notion of approximate "independence" or "uniform initialization" of the ansatz with respect the eigenbasis of given objective observable, as when + 0 and ( -> oc the ansatz is chosen in completely group-invariant way over Aut (A). By Eq.
(45) these designs are (up to isomorphism) direct products e-approximate I-designs over SO (N). SU (N), and Sp (.V).
Before defining e-approximate t-designs we define the Haar measure on compact Lie groups such as Aut (A). This is the unique group-invariant normalized measure on these compact Lie groups
Definition (Haar measure on (Haar 1933)). Let G be compact Lie group. The unique measure satisfying:
| dp=
(46)
as well as
du = d (gpu)
(47)
for all g G is the Haar measure on G. The existence and uniqueness of this measure follows from Haar (1933).
We now define e-approximate t-designs. We will here use the "trace norm definition" out of conve- nience, though all of the commonly used definitions are roughly equivalent (Harrow & Mchraban 2023
Definition 10 (c-approximate 1-designs over G). Let G be a compact Lie group with defining rep- resentation over N-dimensional space, and f the Haar measure over G. A measure satisfying:
Σ [Bu [m (U)] Ey- [m (U)]] S e,
(48)
where the sum is over all degree-(1,1) monomials m (U) in the entries of U and U*, is an 6- approximate t-design over G.
A.4 WISHART MATRICES
Our main results will be given in terms of Wishart-distributed random matrices, so before proceed- ing we give a brief review of this distribution. We will use W (r, Σ) to denote the 3- 3-Wishart distribution (Dubbs et al. 2013), where:
3 1, 2, 4 indicates it over field IR, C, TH, respectively;
Nj is the degrees of freedom parameter of the distribution;
Σ E 75" (R) is the symmetric, real-valued, dositive-definit scale matrix parameter of the distribution.
When we do not specify 3 we implicitly are referring to the usual Wishart distribution where / = 1. The 3-Wihhrrt distribution is over positive semidefinite matrices of rank r, constructed by drawing i..d. standard Gaussian entries over F:
X 8 (0,
(49)
19

---

Published as a conference paper at ICLR 2025
Density
Eigenvalue
Figure 2: Marčenko- Pastur distribution. The density of the Marčenko- Pastur distribution- asymptotic empirical eigenvalue distribution of normalized Wishart matrices- the regime where < 1 and where y = y = 1 the associated Wishart matrix transitions from being full-rank to being low-rank.
for E [n] and j E [r], and considering the distribution of:
W v E'XXVE
(50)
Here there exists the so-called Bartlett decomposition for W ~ W (r, I,)::
W W~VELL
(51)
where L is lower-triangular. When 2 n, the the entries below the diagonal of L are i.d. 2 (0, d-dittrrbbe and the diagonal entries are i.i.d. distributed as (Rouault 2007):
BTg (B - i 1)),
(52)
i.e., the L, are x-distributed up overall scaling of 3- When r n, L is with first rows above and all other entries i.i.d. according A/ (0, 1) (Srivastava 2003 Li & Xue
2010
Yu et al. 2014). This completely characterizes the marginal distribution of the entries of any I.))
W
Our final results will be written in terms of real Wishart matrices W. i.e., those with 8 = 1. When we analyze the asymptotic density of local minima of a JAWS it will turn out we will need to consider the asymptotic spectrum of a real Wishart matrix, which we now discuss. When the scale matrix of W is the identity, as n,r -> Do with = 7 held constant, the spectrum of r known to almost surely converge weakly to fixed distribution (Marčenko & Pastur 1967). This fixed distribution is the Marčenko-Pastee distribution, given by
duMe dd
(1- (A) (1 11+ Vl++ A) (A (A-7-), 6 SAS
(53)
2
where
(1+
(54)
and the indicator function. This distribution is illustrated in Figure 2
A.5 CONVERGENCE OF RANDOM VARIABLES
We finally discuss in more detail the various of notions of convergence of random variables that use throughout our paper. Given a sequence a set (X:"), of real-valued random variables as N oo, we say (XN); weakly converges (or converges in distribution) to (X?"), if the joint cumulative distribution function FN of the X converges the joint cumulative distribution function F of the X at every point at which proo is continuous. That is,
lim
FN (x) F.0 (z)
(55)
N N0000
20

---

Published as a conference paper at ICLR 2025
for all points at which F is continuous. We denote this at the level of random variables with the notation:
(x*), (x;));
(56)
One way to quantify the rate of this convergence is through the use of the Lévy- Pro-hPro metric. Letting nd and pd be the densities associated with FN and F*, respectively, the Proy-orro metric is given by:
F (Ar) +c]. a pp<) = inf (c VA, FN (A) F
(57)
where here A is c-neighborhood in oc-norm of A. pp + 0 if and only if (X;"), (x
Convergence in distribution is closely related to the pointwise convergence of probability densities. Indeed, the latter implies the former by Scheffé's theorem (Scheffé 1947). The former implies the latter if the pd are equicontinuous and uniformly bounded as N + oo Boos 1985
We finally discuss convergence in probability. This is the statement that, for all 0.
lim P -
10 c) 0.
(58)
N-000
We denote this convergence using the notation:
(xx), 2 (x.);
(59)
One way to quantify convergence in probability is through the Ky Fan metric, which is given by: (p p inf {c 0 (|| 1
S {
(60)
(p p000 O if and only if (xM); 3 (x.)). The Ky Fan metric upper bounds the Lévy- Prokhorov metric (Strassen 1965). consistent with the notion of convergence in probability implying
convergence distribution.
We will in what follows often be loose with language and say two sequences of distributions con- verge to one another rate K, either weakly or probability, rather than state convergence to a fixed distribution. This should be understood to mean that their distance in associated metric asymptotically vanishes as K-1
B FORMAL DISCUSSION OF THE MAIN RESULTS
B. THE JORDAN ALGEBRACC STRUCTURE OF VARIATIONAL Loss LANDSCAPES
With these preliminaries in place we can write the variational loss function given in Eq. 36 as the canonical trace form on H (C):
L (0: - (P.T
(61)
where here p, O E H. (C), and (()) parameterizes some path-connected compact Lie group T. Note that (T (0) O}e generates a Jordan subalgebra A H (C) such that O E AA and
T C Aut, (A). As T' (0) E A for all 0, the trace form is zero for components of orthogonal to
A. Because of this we will often also consider p an element of AA slight abuse of notation.
We call a variational loss function of this form a Jordan digebra-supporter ansatz (JASA), in analogy with the term Lie-algebra supported ansatz (LASA) introduced in Fonnann al. (2024). However, whereas variational loss functions are LASAs only if iO belongs to the dynamical Lie algebra generating the ansatz, all variational loss functions are JASAs (up to assuming the path- connectedness of T). Indeed, in Appendix H we give direct mapping from both LASAs and the variational matchgate formalism of et al. (2023) to JASAs and show that a mapping in the other direction is generally not possible.
We are often interested the case where our variational loss landscape has some sort of structure, ie., when A = +., Aa * N (C). In Appendix/Gwe demonstrate that in the context of examining variational loss landscapes, the spin factor sectors (AA,, = LN.) effectively reduce to real symmetric (A,, H. (R)) sectors. Because of this, will from here out focus the case when A., is
21

---

Published as a conference paper at ICLR 2025
the Jordan algebra of Hermitian matrices over field, i..,, Aa = H.. (Fa) for RR We will also throughout use B. = 1,2, 4 to label these three cases, respectively. as is commonly done in the physics literature. We use Tr, (-) to denote the trace of in the defining representation of A. and Tr (-) to denote the trace of according to its representation H (C) 2 A.
Recalling from Appendix A.2 the universality of the trace form and the direct product decomposition of Aut (A), we can rewrite the loss function of Eq. 61 as:
l { p) = Σ ea (0: p)
E (p" T (p=)T=(0)0))
where here ." is used denote the component of in Aa, T" is the canonical trace form Aa. and In > 0 is the constant due to Eq. (42).
We now give an explicit form for T (0). Recall that elements of Aut, (Aa,) are, in the defining rep- resentation of Aa. given by the conjugation action by elements of SO (Na). SU (Na), or Sp (Na) when F, R, C, H., respectively. We let G, denote the corresponding Lie group and Oa the asso- ciated Lie algebra. We also denote G as the direct product over G.. and g the corresponding Lie algebra. Following the typical structure of variational quantum algorithms (Peruzzo al. 2014) will assume 7 (0) in the defining representation corresponds to conjugation by:
[ 1=1
U (0) (0)= 90
exp (0.4)) gi h² Go
(63)
for some Jt, h E G and A, E g. where denotes the projection onto Go or ga appropriately, With this choice of ansatz it is also easy to the derivatives of & have algebraic interpretations. For
instance at 0 = 0, 2.121. j.j.
(,: p) = Elar (88 (
(08p"9n"
gn (o at , hon"
(64)
of 0,0, (0: = Σlat" (28 p0 90
9f gtt )
(65)
where the commutator action of the automorphism group of a Jordan algebra is canonically defined through its representation L defined in Eq. (3)) (see Lemma of Koecher (199b)).
B.2 JORDAN ALGEBRAIC WISHART SYSTEMS
We have demonstrated that all variational loss landscapes are Jordan algebra-supported ansatzes (JASAs). We are finally ready to discuss our main results, which give explicit expression for the loss landscape of a JASA when its ansatz takes the form of Eq. 633) To do this we first define Jordan algebraic Wishart system (JAWS), leaving for now ambiguous the connection to Wishart matrices.
Definition 11 (Jordan algebraic Wishart system), Let O belong to a Jordan algebra A C H (C) and let T be path-connected subspace T C Aut, (4). Let
A Aa
(66)
be the decomposition of A into simple Euclidean components as in Theorem S with associated
decomposition T ~ X Ta as in Eq. (45). Let G Ha be independent distributions over T and
A {AiPe- elements of the Lie algebra of Aut, (A). We call
J == (A. T..... 9 H,
(67)
a Jordan algebraic Wishart system (JAWS).
Every JAWS has associated with a loss function discussed in Appendix B. Using .a to label the projection of into Aa in its defining representation, the associated loss function takes the form:
e e p) = Σ (0:p) = Σ la Tr. (~ ( (0) (0)')
(68)
22

---

Published as a conference paper at ICLR 2025
where U (0) Ga for all f and is of the form:
P
U (0) = To
oI exp (0.A of
h²
(69)
Here, of is the defining representation of element drawn from from the distribution G and ha is similar for H!"
Our main result concise, asymptotic description of / (0: p) when the ansatz is chosen "suffi- ciently independently" from O (up to respecting the algebra A). More formally, the rate at which i {0;; p) converges its asymptotic limit depends on the parameters defined by the following as- sumption.
Assumption 12 (e-approximate t-design of observable basis). Each H. is an e-approximate t- design over Ta Aut (Aa).
If there is sense of geometric locality in the system, i.e., if the reverse lightcone of O is of the form:
U (0) (0) Ō 6
(70)
for some Ō acting space of dimension most D exp (O (p*)) for constant geometric
dimension d, one can explicitly enforce this with only modest overhead in circuit depth. Indeed,
may embed Aa into qubits and the construction of Harrow & Mehraban (2023) give
a d-dimensional random circuit of depth O (poly (1) that achieves this over Aut [ (C))
Aut (Aa,) for some = O (p") and e exp (- Ω (p)).
Given Assumption 12 we are able to prove our main result on the convergence of loss functions. We are able to prove our convergence for any N-dependent overall normalization of the loss function N. the only requirement being that central moments sufficiently large (constant) order have finite limit N + oc. This is always true for N = 1 and, depending the specific choice of t-design, potentially holds for any N S O (N2-6) where > 0 is constant.
Theorem 13 (Loss function distribution). Let I be a JAWS satisfying Assumption 72 with loss function & (0; p) for p E R. Further assume that
Tr (100)²) =0
(71)
NO.999
Fix O. Assume N < O (min, No.99) is such that there exists constant k" such that the central moments of order k > K- of (Ne (0; p)) PER are finite as all e t,,,,,,, Na + cc. We have the convergence in joint distributions:
(WE(O:P)) pER ~ (NE((::)) PER
(72)
as all E 6-,,,N + oc, where
= Σ La Tra W"",
(0:p) = Σ jax (0:p) (0:
(73)
Ta
0a is the arithmetic mean eigenvalue of 0 in the defining representation:
Jth = N- (0%),
(74)
and the aa are independent Wishart-distributed random matrices with Bara degrees of freedom, where
0 1
Fa
(75)
0 1
In particular, the distributions differ in Lévy Prokhorov metric by
log log (W-
log (t)
Wlog (Na)
=
+
Σ
(1).
(76)
Tog (N-1c-1)
Vt
N1.001
23

---

Published as a conference paper at ICLR 2025
In Appendix [ we show that Eq. (71) can effectively be taken WLOG. This is because all p" for which this is not true are such that &x (0: p) have asymptotically vanishing variance even when rescaled by the potentially exponentially large / = O (N). This is why do not take Eq. (7I as a numbered Assumption. Examples of the loss density are illustrated in Figure Ta).
of pointwise vergence densities given equicontinuity and boundedness of the appropriately normalized loss function density. Whether this is true depends on the specifies of the distributions 9, H. particularly whether the loss equicontinuous and bounded at the same N VTa scale l We give some standard examples where this is true in Appendix K
We can strengthen the statement of weak convergence in Theorem 13
Corollary 14 (Convergence of loss function densities). Let J f, be as in Theorem
13
Assume the co. Then the
density of (vral" (0: P)) PER.a equicontinuous and bounded as all c c,,,, ,t,No No
joint density of (Vral" (0: p)) PER,a is pointwise equal to that of (vral (0: p)
up to
PeR.a
additive error O (T).
As-written our result only gives the loss function distribution at single point 0 in parameter space. However, any finite set @ points parameter space can be considered by taking
P (0) = U (0) p (0)
(77)
to be elements in an augmented set of input states:
R' R x e.
(78)
We give more concrete form for the parameter-dee -ereedeee-deeeedencce of the loss landscape by considering the joint distribution of with its derivatives. We begin with the gradient. In order to consider the joint distribution over what can potentially be many gradient components-a number growing with the number of parameters p-we take the following additional assumption on the growth of I with so we are able to fully capture correlations between the derivatives.
Assumption 15 (Scaling of parameter space dimension with (-design). The number of trained pa- rameters p satisfies
log (c-1) Vt log log (E- log (t)
v=
)))
p² so min
(79)
where the G are i.i.d e-approximate t-designs over Ta Aut (Aq), and
p s O [poly (min No))
(80)
is possible weaken this assumption and instead assume that the 9 are c-approximate 2-designs rather than i-designs, but this at the expense of requiring N s o [ and only considering jointly at most log (N) components of the gradient. We leave further details of this alternative setting to Appendix[] []]
It will also be convenient for the rest of our discussion to consider concrete set of A,, Assumption 16 (Concrete choice of A:). For F,, C, HI, the A are rank-1. For F. = IR, the A rank-2.
The distinction between the cases F,, = C, and F. in the defining representation of so ).). This choice of A may seem unphysical due to their nonlocality, but one can emulate the behavior high-rank (e.g., Pauli) rotations by considering a factor of Na more layers Pa in given simple sector-each, each, for instance, performing rotation under each eigenvector of A and then tying the associated parameters together. This breaks no other assumptions as taking P. - NaPa maintains Eq. (80).
due to there being no rank operators
We then have the following theorem. It is stated assuming only a single input which is rank-1 1 when projected the defining representation of each simple sector to simplify the final result. However, in Appendix D33 we give a full, exact expression of the joint distribution in terms of Wishart matrix elements for any choice of R e p and independent of the scaling of N,, Ta-
'ee here use bound on p² so that later we reuse this assumption for the Hessian; at this stage really only need this bound for
24

---

Published as a conference paper at ICLR 2025
Theorem 17 (Gradient distribution). Consider the setting of Theorem 13 with the additional As- sumptions 15 and 16 Assume =2 is bounded oe the standard deviation of the eigenvalues of and assume 'Ri = 1 with element p such that each pe is rank-1 in its defining representa- fion. We have the convergence joint distributions:
2
p) (0; p)),E(! ~ (NE( (0:P)) iE(p]
(81)
as all E -1 ,,,,No + oc, where conditioned on
E (0:0:) p): =
(82)
have that
20 Tra (p'))
Baza
3/220 (0:p)
V
In Tn Gi
(83)
Na
where
= li (0:pp) &g (0: f (0, (0,)) = (0,0)=
():
2 >Ep p): p)
(84)
Here, the Xi are independent diditribbtted random variables with max (2, Ba) degrees of freedom and the G are i.i.d. standard normal random variables. In particular, the distributions differ in Lroo-oro metric by
plogg log (W- +
p plog (t)
Wlog (Na) N1.001
N log (N) (1) (85)
T' =0=
+ Σ
log (N-
V
N 1/
This result is stronger than typical barren plateau results as it gives the full distributional form of the gradient- eeen when conditioned on the loss contribution 2"-rather than just its variance (Mc- Clean et al. 2018 Larocca al.) 2022 Fontana et al. 2024 Ragone et al. 2024). The conditional
distribution as in Eq. (83) is illustrated in Figure b)..
Just as with the loss function, we strengthen Theorem to show pointwise convergence in probability densities assuming equicontinuity of the original distribution.
Corollary 18 (Convergence of gradient densities). Let J. &, be in Corollary 14
Assume the
density of (Vraly ta (0; p)) is equicontinuuus and bounded for all z" E Rzo as all e .t, Na + co.
Then the joint density of (yraly (0; p)) is pointwise equal to that of (Vralz (0: p)), up to additive error O (TT).
We now give our final result, which specifies the joint distribution of not only the loss and gradient but also the Hessian at critical points. This is required to reason about the critical point distribution of the loss landscape using the so-called Kac-Rice formula (Adler & Taylor 2007) that we will
discuss in detail in Appendix C.3 We state our result assuming a, < ō for simplicity, giving as we
did for the gradient the full expression in terms of Wishart matrix elements in Appendix
Theorem 19 (Hessian distribution). Let J. t, é be as in Theorem 17 Assume all 5t (1) as
N. co. We have the convergence in joint distributions:
(0; p) 0,6 (0:p),0.05 (0; p)) PER (0(0: (0: p) Res (0:p)) pER
(86)
as all -1 ,, Na + oc, where conditioned on
{0: (0: p) z²
(87)
and
2 (0: =
(88)
we have that
for 2u, (0:p) = 2I 0. Tra (pa)
2a
G Xx (i) Wa (i),
(89)
N2
I.oa
25

---

Published as a conference paper at ICLR 2025
JAWS
Shallow
Drap
Underparameterized
Overparameterized
Grotlents
WHRFs
MG+
QNTK
Figure 3: Relation between models for quantum variational loss landscapes. Our introduced theory for the loss landscapes of quantum neural networks (QNNs) are Jordan algebraic Wisharr systems (JAWS), which relate the algebraic structure of given QNN architecture random process description of the loss landscape. This JAWS description reduces to the previously- studied Wishart hypertoroidal random fields (WHRFs) and quantum neural tangent kernel (QNTK) in different settings, and also reproduces the loss function variance known for Lie algebra-supported ansatzes (LASAs) and matchgate (MG) networks. Cartoons of the loss landscapes associated with previously studied models are shown.
asymptotic
where
(a)(0:p) (0 (0.p) === (L:( iElpl = =0] = E (0:p)
Σ
Ea " .0 (0; p)
(90)
Here, the G i.i.d. standard normal random variables, the xi independent x-distributed random variables with max (2, Ba) degrees of freedom, and the Wa are independent Wishart- distributed random matrices with Bara degrees of freedom. In particular, the distributions differ in Lévy-Prokhoron metric by
n²" =0
log (1(1logg (N-4-11) +
pd log (1)
N log (Na) -0
)
(91)
log (N-
V
Na
C FORMAL DISCUSSION OF THE CONSEQUENCES OF OUR RESULTS
Before proceeding with proofs of our results we examine their implications. These are summarized Figure
C.I BARREN PLATEAUS
The first implication our results that we will discuss is the unification of barren plateau re- sults Foontan al. 2024 Ragone et al. 2024) in the large Na limit.
Corollary 20 (General expression for the loss function variance). Let & be as Theorem 13 The variance is:
Tr ((0m)) Tr (p))))
Var [e (0:p))
Σ
+ 0 (T).
(22)
ding (0n)
26

---

Published as a conference paper at ICLR 2025
Here, dimg (ga) is the dimension of the automorphism group of Aa and is in Eq. 76
Proof. Note that
Na (Na -1)
Na
1) Na dimR (gn) = (Ba, (11--1)
ON.).
(93)
2
Similarly,
(om)² 2
10
(94)
Ta
N2
By Eq. 42 we also have that
La 1 = (109)²)
(95)
with Tr (.) the trace on the full N-dimensional Hilbert space. Using the fact that the diagonal entries of Bn W are i..d. x2 -istttrrbttted with Bara degrees of freedom (see Appendix A.4D. we then have from Theorem 13 that as Na + 0c:
Var [e [0: = Σ Var ( (0: + (z)
21 a (09)2
Tha (109)²)+ + (0)
Bara
2Ia (on)2
(96)
Bara Tr ((D9)²) +0(TT)
Tr (10n))) Tr (pp)32)
Σ
+ (T).
dimg (ga)
a
This result immediately implies Theorem of Ragone et al. (2024) in the case of Lie algebra- supported ansatzes. However, our result holds for all variational ansatzes, i.e., does not rely on either i0 or ip being a member of the dynamical Lie algebra generating the ansatz.
In the language Ragonee et al. (024)), Tr (p²)²) is the Ao -purity PA, of O, and be thought of as measuring the generalized entanglement (Barnum et al. 2004) of p with respect to the Jordan al- gebraic structure of A. In this sense, this barren plateau result can also be thought of as generalizing the eutanglement-induee induceed barren plateaus previously studied by Ortiz Marrero et al. (2021).
C.2 THE QUANTUM NEURAL TANGENT KERNEL
We now connect our results to the quantum neural tangent kernel (QNTK) literature (Liu et al.,
2023 You et al., 2022, Garefa- Martin et al. 2023; Girardi & Palma 2024| Gareia-Mann 2022
et al. 2024 The landmark result in this field is that, under certain conditions on R, O, and A, variational loss functions are asymptotically a Gaussian process when e - N + xc. However, this same body of work has noted that such a Gaussian process description cannot generally be true: for instance, if O is rank-1 and A is the space of complex Hermitian matrices, the loss should be Porter-Thomas distributed as this reduces random circuit sampling setting (Boixo et al. 2018).
Our results can be seen as a unifying model of neural network loss landscapes, including both when convergence to a Gaussian process is achieved and when it is not. Recall that Theorem 13 demonstrated that the asymptotic expression for the variational loss with objective observable 0 is distributed as a Wishart process:
Ia Ta Tra Wp*
(p)=
-
Σ
(97)
TThough careful examination of our proof this error is actually identically zero when t 2 2 and Assumption a2 here only taking second moment.
0 in
27

---

Published as a conference paper at ICLR 2025
when rescaled by quantity W exxonentially large in the problem size. This correctly captures
the Porter-Thooe Thomas behavior when A is the space of complex Hermitian matrices and rank (0) = 1. This is because the diagonal entries of a complex Wishart matrix x (ddistributed with two degrees of freedom each, which is identical to the exponential distribution
Indeed, more general result can be used to exactly characterize when QNNS asymptotically form Gaussian processes. This occurs when the loss is scaled by some N such that Eq. (97 has nonvanishing variance yet the higher-order cumulants vanish. We compute the third-order cumulant:
R3 (NE (p) (No(
=0 o 1
13 ( T
Tra (109)))
(98)
We thus can state this pair of conditions formally as follows.
Corollary 21 (Exact conditions for convergence Gaussian process). Let J E. be as in Theo- NE (0; p) is asymptotically a Gaussian process as c ,t, oo if and only if
rem 13
N
min
(99)
O Tra (loaju)& 3
and N2 Var [E (0: (with the variance as given in Corollary 20 is nonvanishing for all p E R.
When these conditions met, the covariance is given by:
K. (p) lim KN (p. pp)
N
Ba TT ((04)²)
lim N² Σ
I, (Tra (p" -y)) Tra (p" (Wa ra))].
N
2 dimz (ga) ra
(100)
now easy to see why (for simple A) the case does not converge Gaussian process: when r = we may only choose a normalization N < (N) such that higher-order cumulants vanish asymptotically, but then the variance vanishes asymptotically. strate convergence to a Wishart process through any finite number of cumulants, not just the first two.
contrast, our results demon-
We can also be more concrete the form of K. (p. p'). Indeed, we are able to show that each algebraic sector Aa contributes to the covariance an explicit, quadratic expression in pa and p²a depending only on the field Fa associated with its defining representation. In other words, whether or not a given quantum neural network asymptotically a Gaussian process just depends on how SWAP tests of the inputs scale. As the expression is complicated we do not reproduce it here, instead giving references to where it may be found for each of F. = R, C, H.
Theorem 22 (K quadratic p. p'). K. (p. p') can be written as a closed-form, explicil function of only the (p*), Tr (p' Tr (p))²). Tr ((p")))) and (*))
Proof. By the Haar invariance of Wa there are explicit formulas for covariances of the form of K. (p. p') degree-2 polynomials p and p'. These formulas given in:
1. Theorem of Redelmeier 2011 when F R;;
2. Theorem 3.5 of Mingo Speicher (0066) when
3. Sec. 5.3 Redelmeier 2221 when HI.
The result then follows by noting that T, Tra (.0) Tr (r())..
It is also easy to check when = 1 and the ansatz unitaries are Haar random that the assumptions we proving our results hold for any A (N).
28

---

Published as a conference paper at ICLR 2025
We give simple example of the calculation of I. when all inputs Po can be mutually diagonalized. By the unitary invariance of Wishart matrices we can assume WLOG that all inputs are diagonal. Inputs are then completely parameterized by the cigenvalues a RoO the input states, where ||l|| = 1. By the independence of diagonal entries of a Wishart matrix we then have that (when ==1::
Ba Tr (00)22 KN (2, N²
N Σ
Σ
Var (1 W
[8] +
dimæ (0a) ra i=1
(101)
Tr (100)2)
N2 Σ
x x +
dimR (0a)
where in the final line used similar simplifications in Appendix C.1 Noting that x² x/ is just the overlap of p" and p yields an equivalent formulation:
KNN (p. p¹) = = Tr (0a)²
Na
Tr (0()))))
Tr (pppp))) + (1).
dim (ga)
N
(102)
In particular, when A is simple, a N = Ω normalization is required for the network to asymptotically form a Gaussian process over pure inputs.
While previous results have shown that Gaussian processes efficiently train, we argue that this may paint an overly optimistic picture for generic QNNs. Focusing on the case where the ansatz an θ (log (n))- 2-dimensional circuit on n qubits, we have that the QNN forms a Gaussian process when:
=
= exp ( (log (n)²))=
(103)
i.e, at normalization superpolynomial in n. Thus while it is known that such networks can achieve constant improvement in Á in time polynomial in via gradient descent (Girardi & Palma 2024), this translates to a superpolynomially vanishing improvement in the loss function value when in the physical normalization of V = 1. More generally, the covariance
here derive can be used with results in the classical neural tangent kernel literature to reason about the training behavior (Neal, 1996 Choromanska al.) 2015 Chaudhari] 2018 Lee al. 2018) and generalization ability (Jacot et al. 2018 Wei et al. 2022) of this class of quantum neural networks, which we hope to analyze in more detail in the future.
C.3 LOCAL MINIMA
We end by examining the distribution of local minima of the loss landscape. This has been done pre- viously in the more restricted setting where no structure imposed on the loss function (Anschuetz, 2022 Anschuetz. & Kiani 2022). In this section we assume the assumptions of Theorem 19 and that as ,, No 00,
Pa
Ta Bara
(104)
held constant. Here, Pa is the number of A, that are nontrivial on the simple component A.. of A. Yo is the so-called overparameterizaation ratio discussed in Anschuetz (2022); Anschuetz & also assume for the simplicity our expressions that each simple sector is fully
Kiani (2022). controllable, i.e., that each ansatz generator A; is only nontrivial A. In principle a more general expression could be written as well, though the gradient density would be convolution over complicated probability densities, and the Hessian distributed as free convolution of multiple Wishart matrices.
single simple component of
We calculate the distribution of local minima via the Kac-Rice formula (Adler & Taylor 2007). which gives the expected density of local minima of a function at a function value 2. Anschuetz (022) demonstrated that the assumptions for the Kac-Rice formula satisfied for variational loss landscapes, and when rotationally invariant on the p-torus takes the form:
E (Crto (=)] (2rr) E (det (H.) (H2 Z P) P 0) TP =].
(105)
29

---

Published as a conference paper at ICLR 2025
Here, E [Crty (2)] is the expected density of local minima function value 2, G2 is the gradient
conditioned on = 2, and H is the Hessian conditioned on & = 2 and = 0. In a slight abuse of notation, here P [-] denotes the probability density associated with the event
We can evaluate this expression using Theorems
and
Corollary 23 (Density of local minima multiplicative leading order). Let J, ( be as in Corol- lary 18 and Theorem 79 Assume each ansatz generator A, is only nontrivial on a single simple component of A, and further assume that p" is rank-1 in its defining representation. Assume well that the overparameterization ratios:
Pa
Jaa
(106)
BaTa
remain fixed N - 20. Let MHe be the empirical spectral measure of
= O O
(107)
where (for 2 ST. |G; with random variables defined as in Theorem the Hadamard product. Let X,, denote the infinimum of the support of μHş. Then the expected density of local minima of at a loss function value is:
19 and @ denotes
E E[Crt [z)] = * E [Crth] (z),
(108)
where
p- In (Crtt (=)]) =In Tmax (2, Ba)
1 (1-
2 (1-
Tr (T(ppa) T ))
+
Tr (pa)) +ln
4VBa
27a
duurs (a)
1 (15 2 20] +o
+ p
IE
exp
(109)
Proof. We will first consider single simple component A, with pure p (and with labels implicit for clarity of notation), and describe in the end how a full distribution of local minima can be determined from this via convolution. From Corollary 14 follows that the density of the loss is 8
8r-1
frz 27
frz 27
exp
P 2
(110)
2r
fr
2
Similarly, using Corollary 18 and recalling that 33 is one of 1, 2, or 4, we can evaluate the density of the gradient at zero:
dx f 2 (x) (x))
4 -(m=xx(22)0
[max (2,
G = 0]
(111)
22To==
where fymax(2,a3 the density of a x-distributed random variable with max (2, 3) degrees of free- dom and
2Ta N
B2
(112)
lo
Finally, consider the Hessian determinant. Recall Theorem which gives the Hessian as:
19 for the Hessian components,
N
lI H. -
(113)
210.
where Ši, = G2xi- We now claim that H. can only be positive semidefinite if all G; 2 0. To see this, note that xi W., is always nonnegative. When G, < 0, then, the (i, entry of $ @ W negative and thus is not positive semidefinite. We therefore can consider GG, rather than G, up
Teehhically need to consider the density of the loss rescaled by / to achieve pointwise convergence in densities, but the results equivalent after rescaling
30

---

Published as a conference paper at ICLR 2025
to pulling out a factor of 2 P from the expectation. Putting everything together, to multiplicative leading order as N + oc,
p- In [Crto (z)])
7max (2,
1
In ( (Ta)
4/3
27
To
(114)
+ +p"
In
exp
p du du (2)
1 2 20 +o
This completes our proof for a single simple component A, with pure pa. To calculate the loss landscape of JAWS associated with nonsimple Jordan algebra, note that:
P = * =
(115)
where * denotes convolution. The relative weights of the various sectors introduced by Tra (p*)
can be accounted for by taking:
La + Tro TT (p").
(116)
From Eq. (105) and our simplifying assumptions, then,
E [Crta (z))
* EE [Crttl -T((p) (z)
(117)
where E [Crtā] denotes Eq. (114) associated with the algebraic sector A..
While Corollary 23 is exact, it is obtuse almost to the point of obscurity, particularly due to the ex- pectation over the Hessian. The obstruction to further simplification the presence of the Hadamard product between S and W in Hg which is difficult to handle analytically. get around this, we consider slightly modified quantity where we condition both sides of Eq. (I05) the events:
|G = X
(118)
In effect this can be considered as regularization scheme, where new parameters 0 are introduced Lagrange multipliers with associated derivatives P
i 4= 0,
|G,| Xi
(119)
and we consider sufficiently small neighborhood of 0, 0. In this setting the nontrivial compo- nents H take the much more manageable form:
म = No V.VVVVEV W Σd
(120)
where na is a diagonal matrix with entries i..dd. x >d-dittributed with max (2,3,) degrees of free- dom. Analyzing the expected determinant of this random matrix leads us to prove the following.
Corollary 24 (Density of local minima to multiplicative leading order, regularized). Consider the setting of Corollary 23 conditioned on |G,| Xi such that H is as in Eq. (120). Let paar be the Marčenko- Pastur distribution with parameter ry. Then the expected density of local minima of at a loss function value is: IE [Crt (z)]
* octT
[CCrt]] (2),
(121)
where:
p p-1 In (E(Crt)) (2)]) = =In
7max (2,Ba)
1
=
+ In
2/BB
2
Tr (p oa
Tr (Tr oa
max (2,
/
1-
2
(122)
9The Kac-Rice formula stated in Adler & Taylor 2007) allows one to consider this modified gradient jointly with the original loss.
31

---

Published as a conference paper at ICLR 2025
Proof. As in Corollary 23 we focus single simple component Aa. E is positive definite with probability 1. As it is diagonal with i.i.d. x" dddiitrrbbtted random variables each with max (2, S) degrees of freedom.
exp f dus (2) exp ( (p max (2, - y1 (m
(123)
2
where y is the empirical spectral distributiin of Σ and is the Euler-Mascheroni constant. What remains be considered is the spectrum of
Br
D2 = = H =N
(Br) W.
(124)
N
Recall that by assumption = = (1) so by, e.g., Lemma 25
(proved in Appendix D.I
Br N
= (1).
(125)
We need only focus (Br) W, then. As discussed in Appendix A.4 this random matrix has empirical eigenvalue spectrum weakly converging almost surely to the Marčenko-Paan Pastur distribu- tion with parameter 7. In principle, large deviations in this convergence-evenn if they occur with exponentially small probability-can contribute corrections to the expected determinant due to its exponential sensitivity on the eigenvalues of D However, we show in Appendix Ithat these large deviations are dominated in the expectation by the Martenko-Pastur distribution. Noting that the Marčenko- Pastur distribution with parameter 7 has support at the origin if and only y 1 and taking convolutions over simple algebraic sectors as in the proof of Corollary 23 then yields the final result.
Dropping multiplicatively subleading factors from Eq. 122), effectively have demonstrated that the density Ka (z) of local minima for a given simple component A. is asymptotically given by:
Pa 27
Kic (2) = fr Tr (p()) Ta
(126)
2Ta Pa
%o < 1, and otherwise:
Ka (2) = (
(127)
We here have used the expression for the density of the gamma distribution:
Jr (x; 0)
(k) 0 k exp
1
(-7)
x
(128)
Convolving Ki over many simple sectors thus yields the final density for 0:
(2)
* fr or Tr
Bara
2
(129)
2 Bara
This distribution illustrated in Figure [C) for various parameter regimes. To multiplicative lead- ing order in r this agrees exactly with the asymptotic local minima distribution studied in (nnnchuuetz (2022 Anschuetz Kiani (2022). See for instance Eq. (1) of Anschuetz (2022), which studies the case A H (C); i.e., one takes Sa + 2, Ta m, and TT + to translate from our setting to their setting.
We can simplify this expression even further by noting that, asymptotically, the convolution of many gamma distributions is also ramma--istribon by the Satch--Sttettwwaite equation (Satterthwaite
1946 Welch 1947).
This yields:
(2) fr (2; ker. Beat
(130)
where
Eangei Tth Tr (p9))
Tr (104)2 Ir ((0z)2)
(Nrr -00lay Cep.
kill
(131)
Eoroel
2(0012 T3
dinig (0a)
Anto
Eayo<1 2( 11p
Tr (10m)²) Ir ((a)))
Oar = Earo< <1 0a TT((p))
Baro
(No No-)
7-1
(132)
up.
dimg (ga)
a.% 21
32

---

Published as a conference paper at ICLR 2025
Here,
Pup
Σ TH 017
(133)
the mean loss function value over the underparameterized sectors, and the limit each line is due to the identities (recalling that here we assume that pa R is rank-1 in its defining representation):
(o 2o") (pt)) (0 =
Tra (109)²) = la (0") Tr (())²)
(134)
la ( La Tra (00)² IaTa Tra ((09)²) Na
la No
Tr (109)²)
(135)
N
as well the identities considered in Appendix
Intriguingly, the relevant features of this density in the underparameterized regime are controlled by the A.- -puritiss of both O and p in the sectors in which they underparameterized. This is the same quantity which controls the variance of the loss function (see Appendix [C.I). However, even when there are barren plateaus in the loss landscape- for instance, if the variance of the loss function are polynomially vanishing in Nhe density s: (2) may still have exponentially small measure near z 0 as the gamma distribution has exponential tails.
We reemphasize that our calculation of the local minima density was performed assuming the vari- ance of the spectral distribution of each 0 (in units of the mean eigenvalue) asymptotically van- ishes. This was also the setting studied in previous work on the local minima of QNNs (Anschuetz, 2022 [Ascchuetz & Kiamil 2022). This assumption allows to dramatically simplify the Hessian to the form given in Theorem 19 Though it holds for low-weight fermionic (Feng et al. 2019) and lo- cal spin Hamiltonians (Erdös & Schröder 2014), it does not hold for the Gaussian unitary ensemble (GUE) or nonlocal spin systems; these systems are also known to have efficient quantum algorithms that prepare their low-energy states, unlike their local cousins (Chen et al. 2024). We hope in the future to analyze whether this property also has an impact on the behavior of local minima of QNNs.
We finally note an interesting connection between our results and algorithmic hardness. We here only calculate the expected density of local minima at a given function value. If the second moment of the local minima density is also sufficiently well-behaved, then it could be the case that the asymp- totic density of local minima has fixed distributional form; this is the case in (classical) spherical spin glass models (Subag (017). The function value at which these local minima proliferate w.h.p. conjectured also hold as a general (i.e., beyond gradient descent) algorithmic threshold, that is, it is generically believed to be the function value at which better approximations to the ground state become algorithmically intractable find. This conjecture is known to be true for the specific case of pure spherical spin glass models (Huang & Sellke, (2023). Studying second moments of quantum spin glass local minima distributions may thus be tractable avenue for studying the algorithmic hardness of quantum problems.
D PROOFS OF THE MAIN RESULTS
D.1 PRELIMINARIES
We now give full detail the proofs of the main results discussed in Appendix B.2 We begin by giving definitions and notational conventions that we will use throughout proofs. Recall that,
given a JAWS J = AO, T..TT. T.9. H. A, O), we are interested in the joint distribution the loss f (0: p) and its first two derivatives over set of input states p E R, where:
E (0: = Σ aa (0:p) = Σ Ia Tra (p Oal (0))
(136)
and
P
[| of exp (0,47)) g h
U (0) = 9o'
(137)
with , ha Ga. and A ga- As previously discussed we will use .a to denote the defining representation of the projection of into A.. As each sector labeled by a is independent we will here only consider a single a WLOG, with nonzero pD and Oa This will also allow us to remove
33

---

Published as a conference paper at ICLR 2025
the cumbersome notation of labeling all objects with the index for the remainder of this section. To further simplify the language, we will use the term "unitary" to refer to "orthogonal." "unitary," or "hyperunitary" in the context of IF R, C, HI, respectively, unless otherwise explicitly stated. Similarly, we will later see that our results hold for any 0; we will thus leave the d-dependence of é implicit from here
notation. In the following 0 < < Na - 1 to denote basis vectors in the vector space on which the defining representation of H (IF)) acts, We will later use (8) (i.e., Latin letters), 0 S i S p denote vectors in the vector space on which the defining representation of H (F acts.
will use |/)) (i.e., Greek letters),
We now detail our choice of A; given Assumption [6] Assumption 16 has nice interpretation as taking the A, to be low-rank (representations of) basis elements of the Cartan subalgebra h S g. Up the adjoint action of G and an overall normalization (which can be absorbed into 0,), then, we can take WLOG:
A, (0) (1) (1) (0)
(138)
when F and
A, i (0)
(139)
when F = C. When F- = H. we will take parameterization of the form:
exp (0,A.) exp (i(") (0) (0)
exp (iefe) (0) (01) exp (kO(") 10) (01):
(140)
that is, we will assume we have full control over the quaternionic phase. We have chosen here for the A; to be i-independent for convenience, moving any i-dependence to the conjugating unitaries of each layer gf E Ga-
Finally, before continuing give convenient relation between
O
(141)
O
and the standard deviation o, of the eigenvalues of O. This relation will be used to simplify some of our later expressions.
Lemma (r and o, relation). Let O be an x Hermitian operator and be as in Eg. 141 Let o be the standard deviation of the eigenvalues of O, and T the arithmetic mean. Then:
N
(142)
Proof. Let URMS be the root mean square of the eigenvalues of O. Then:
N
N
N
10122
(143)
RRSS
6
D.2 ASYMPTOTIC EXPRESSION FOR THE LOSS
We now proceed with the proofs of our main results. This subsection is devoted to a series of reductions that will allow to consider the hOh! as 3-Wisharrt matrices up to controlled error in Lévy-Prokhorov metric, thus proving Theorem taking the gi to be Haar random, once again up some bounded error in dévy-Pro metric. In proving these results we will heavily rely on various lemmas on convergence in distribution given in Appendix M
I Along the way we will also prove reduction to
34

---

be Published as a conference paper at ICLR 2025
D.2.1 REDUCTION TO HAAR RANDOM h,
We first argue that, under Assumption 12 the h in Eq. (137 can be assumed be Haar random over G up to some bounded error in -oyy-Proooooov metric. We also show that this remains true when scaling by any choice of N -edependent normalization N with the only requirement being that central moments of large (constant) order have a well-defined, finite limit as N - oo; for which maximal choice of N this true depends on the exact form the e-approximate I-design, but is always true for N = 1. In the following we implicitly consider sequence of objects as ,,, co.
Lemma 26 (Weak convergence to Haar random h). Assume forms an e-approximate t-design. Let La, i [d] be multilinear functions of the form:
L = Tr (MhOh"),
(144)
where the M, have bounded operator norm and h ~ H. Let L, be the same with - h, where h is Haar random. Assume any N N-dependent normalization N 0 (N), and assume that there exists a constant Ee"' such that all central moments of constant order > ke" of the N Lg have a finite limit as ,,, + cc. Then the joint distribution of N
NNL, L, differs from that of N Lng by an error at most
d'log log (N- dlog (t)
Ti = =0
(145)
log (N 11-
Vt
in Lévy- Prokhorvv metric as c_1, Nt,N-+ + + co.
Proof. that:
As the L, - E [L,] are bounded random variables, there exists some constant
such
II (L. (L.-E(L.)) 1 (CVRk) LiE
E
(146)
for all index multisets st of cardinality k. By the given assumption A similar bound also holds
for central moments of the A, of sufficiently high order. The bound / = (N) ensures that this
central moment bound also holds true for the NL, : by sub-Gaussianity (Meckes
2019 Vershynin
2018), and also ensures that the μ of Corollary 338 subleading to Eq. 145) The conditions of
Corollary 38 are then satisfied and the final result yielded.
By incurring this error in Lévy-Prokhorov metric we now can assume that the (given Assump tion 12) are Haar random when considering just the distribution of the loss (so d = 1 in Lemma 26 When considering the loss, gradient, and Hessian jointly, the addition of Assumption 15 means that we can assume that both the h and the gi are Haar random. We note, however, that up requiring worse bound on V (i. < o ( the g: being drawn from e-approximate 2-design suffices.
This argument is given in detail in Appendix
When the h and gi reduced to being drawn i..dd from the Haar distribution it is apparent that the joint distribution of the loss with its first two derivatives is invariant under translations of the parameters. This justifies us fixing f 0 in the sequel.
D.2.2 REDUCTION TO GAUSSIAN h, 9i
We now show that certain marginal distributions of the the entries of /Nh, VN g. can be ap- proximated random Gaussian matrices over IF to small error in Ky Fan metric whenever N < O (N). This will allow us to replace vNh, /Ng. with random matrices with i.i.d Gaussian entries, simplifying our results further.
Lemma 27 (Convergence in probability of Haar marginals Gaussian matrices). Let Li, i E [d] uniformly bounded multilinear functions of the form:
functiion
L (Mhhhh!),
(147)
where there exists some D such that
N
(p $ D =
(148)
log (N)
35

---

Published as a conference paper at ICLR 2025
Assume well that d O (poly (N)) and the M, are independent from the h. Let Lg be the same, where now the h are random matrices G with i..dd standard Gaussian entries over F. For sufficiently large N, the joint distribution of NL; differs from the joint distribution of &4 L, by error in Ky Fan metric of at most:
VNDlog (N)
W
03 =0
+
- (1)
(149)
D
for all N O (N), Alternatvely, rank (p) D.
VNDlog (N)
03 = 0
(1).
(150)
N
Proof. We assume p has trace norm 1 by absorbing Tr (p) into M,. We first argue that the high-purity condition is assumed then p has approximation small error in trace norm that is low-rank, To see this, let r; be the eigenvalues of in non-increasing order. If
N
Σ
Ti 2
(151)
D+1
D=1+1
then it must be that p has low purity, i.e.,
D+1
1
Tr (p²) S
(152)
(D+1)
D+1 D
breaking our assumption the purities of the inputs (Eq. (48)). Thus,
r
(153)
D+1
In particular, has rank-D D approximation that agrees up to an (b)) additive error in trace distance.
Consider now the low-rank case. Let
Dln
(154)
N
6 <<< for sufficiently large N by the assumed scaling of D. Following the proof of Corollary 1.1 in
Jiang 2010), then, for sufficiently large N and >>>>>>>> 10 Na5 N
N V
N2 exp (- (log (N)))) exx (- (ND)) )- (155)
VNhi, Gi.j P max iE(N
EE[D
The right-hand side decays superpolynomially with N whenever N S O (N). This implies the emror in Ky Fan metric dominated by the cutoff at / = error from the initial low-rank approximation yields the final result.
This in conjunction with the N
We will mostly be concerned with the specific case when = N0.900 yielding for < (NV099))
(
(V
VV log (N)
0g = -
(1).
(156)
N1.001
36

---

Published as a conference paper at ICLR 2025
D.2.3 REDUCTION TO
O
We end with a reduction more convenient form for the spectrum of O. Let (1) be the eigenbasis of O. Consider the Hermitian Ó:
(a) (nl,
(157)
μ=U
where
O
(158)
| F
is assumed be an integer and (for ō the mean eigenvalue of o)
|0||
No r
k
(159)
By construction
Tr (O) = Tr (Ō)
(160)
Tr TO")) = Tr 0
(161)
We claim that replacing O with Ô incurs only vanishingly small error in Ky Fan metric. Intuitively this follows from the matrices as a single Wishart random matrix (Khuri et al. 1994). We formally state this as the
approximation of weighted of Wishart random
following lemma.
Lemma 28 (Reduction to semi-isotropic O). Ler L,, i i [d] be of form:
= Tr (M (0-
(162)
where VNX has i.i.d. standard Gaussian entries over F, M, is independent from X with bounded trace norm, and d = O (poly (N)). Let L, be the same multilinear functions, where instead of O one has Ô defined in Eq. 157). The joint distribution of NL, over X differs from 0 by error at most
NA log
dN NA
a4 0
(163)
N
in Ky Fan metric, where
6²
4 |01|
(164)
|0
Here, où is the ith eigenvalue 0 values of O, and the mean.
O in non-increasing order, oo the standard deviation of the eigen-
Proof. and Ô can be assumed to be diagonal WLOG as unitarily invariant. Therefore,
X 0 δ x can be written weighted of standard 3-Wishart matrices each with single degree of freedom:
x x xt
1 N Σ (o. ōz) (x:)) (rrel,
(165)
where (a;)) is the (unnormalized) ith column of VNX and o;, Ô; the eigenvalues of O, Ô, spectively, in nonincreasing order. Note that for any normalized (u), (xx]/1)) is gamma- distributtdd Thus, from generalization of Bernstein's 's inequality (see, e.g., Theorem 2.8.1 of Vershynin (011)), for any In),
N
Σ
(a σ.) 1Kss
2
(166)
< 2exp
-c.N min
max; 1os
ōzl
N
(0; - o,)²
N
-i=1
37

---

Published as a conference paper at ICLR 2025
for constant 0. Due to the equality of traces of O and Ô calculate:
1
1 Tr ((0-o)²
0. - o.)²
N
N
2
Trr (0+) Tr (0o)
N
(167)
(Tr (0 - Tr (0) A + ko,+1)
N
00
r
Or+1
0
N
Furthermore,
max |o: - 0i O (0) k) 0 (0r41)
=0
0
- 01
0
0p+1
(168)
0
1
|0 op
1
op
0
The desired convergence follows by the union bound and taking
N
We now claim these suffice to prove Theorem 13
Proof of Theorem 13 For X with i.i.d. standard Gaussian entries over IF and Ô as in Lemma 28
k- XO is 3- Wishar--distrbutedd with r degrees of freedom. The result then follows from Lem- and 28 applied in sequence, with a final error in dévy-Prokhorov metric of:
mas
26
log log (W- log (N-
log (1)
N log (N) N1.001
π mı + 03 + a4 =0
(Tog
(169)
Vt
D.3 ASYMPTOTIC EXPRESSIONS FOR THE FIRST TWO DERIVATIVES
We will now move and construct explicit asymptotic expression for the first two derivatives of the loss function. As previously mentioned, we need only consider 0 by Lemma 26 Here we have that the first derivatives distributed as:
a,e = Ir (p g.A.g. hoh"))
(170)
The second derivatives are of the form (for 2 j):
0,0,e = I' (p g | g.A.g g.A.gh
(171)
Furthermore, using the lemmas from Appendix D.2 up to cost:
d log log (N- d log (t) log (W-
N log (N)
a3 + 0k4 = O
(172)
Vi
N
in (évy-Proohooon metric where is either p or p² we may reduce considering:
i I Tr (pxfox.).
(173)
N
t, Tr (px) X,A.X/ 0] xo).
(174)
N2
L.s N3
Tr (px! X,A,X), [xAXX] 0 x)).
(175)
where:
38

---

Published as a conference paper at ICLR 2025
1. Xu is an N x rank (p) random matrix with i..d. standard Gaussian entries over F. and the X; are N x p or N x 2p (depending on the rank of the A,) random matrices with i.d. standard Gaussian entries over F.
2. The initial states p are of rank at most O (No..999).
Note that these quantities depend only M O (rank (p)) rows of Xo and either 1 (F C, HD or 2 (FF = IR) rows of X, We can therefore further simplify this by collecting all of the relevant rows of X X; into single X of dimensions N x (p + M) (or N x (2p + M) when IF = IR), writing these expressions as
i 10 Tr (px'Ox);
(176)
r
f, Tr (xx
To
x (1) (1) X 0 x)
(177)
rN
To
Tr
(xxi
x x
x (2) ( x 6 x
(178)
rN
Here, we took another error of a4 in Prokhorrk metric to change O to
Ô= 14) (a)
(179)
M=0
as in Eq. (157). Though this expression for the full distribution is unwieldy, it completely character- izes the joint distribution of the loss and first two derivatives in terms of a single Gaussian random matrix X. To simplify further will assume p is single state which is rank-1 when projected into the simple sector we are considering, taken WLOG to be (0) (0). We will also assume in the following that fee is asymptotically bounded.
Details of further simplifications vary depending on whether F is algebraically closed or not (i.e., whether or not we are working in R). We will thus consider the two cases separately.
D.3.1 F = C, H
When F = C, HI, we considering the joint distribution of:
i lā
(10) (0) xτxx).
Tr
(180)
ilō
Tr (10) (0) xt
x
x)x) X x
(181)
N
r Tr (10) (0)
10
x
x 1b) (1) x
x]}x). x):
(182)
where now X is a N x (p+ 1) random matrix with i.d.d standard Gaussian entries over F. Here, when F = H. "I" should be taken to mean of j. k, depending on which parameter of Eq. (140) the derivative is being taken with respect to. We take the two contributions of quaternionic phase to be equal in the second derivative expression as, by the anticommutivity of quaternions, this pression is otherwise identically zero. In particular, this abuse of notation will not change any of the following analysis unless specifically stated otherwise.
Recall that
Ô=
Σ ==
1) (ml.
(183)
Let C' be the complement of Ô, i.e.
C= Σ 1p) (p).
(184)
We have slightly abused notation here; the (i), (i) here are taken to he orthogonal to the nonzero eigenvec-
tors of p.
39

---

Published as a conference paper at ICLR 2025
In partieular, C' and Ô sum to the identity:
C' C+0 IN.
(185)
Furthermore,
W = = V=X
(186)
(187)
are each 3-Wishart-distribute matrices; the former with r degrees of freedom, and the latter with -. The independence of W and V can be checked from the associated Bartlett decompositions. We thus have that the distribution we are interested in is the joint distribution of:
lo W 00), (0)
i
(188)
210
210
4
Re
(i W ())
(1)
X (0))
Re (i (0) W (1) (1) V 10)
(189)
NN
rN
and:
10
(0)
x
X
x 01| x 10}
r
To
NN2 (0)
x
x
x ox 10)
To
rN2 [0
[i] (j) x 10)
To
X
()) X ()) (1) x 10)
rN2
10
(0 x (0) (g) Ox (1) (i) x'x 10)
(190)
TN2
1o
(W + v) (1) (W V) (1) (il W 10)
rN2
To
[0) W (0) (i) (W + v) (W +V)
rN2
To
[0] + +V) 11) (1) W 15) (W +V)
rN2
lo
[0) (W + b) G1 W ( (i) (W v))0)
rN2
We can simplify the expression for the second derivative by grouping terms by their order in W. Note first that all terms cubic in matrix elements of W sum to zero. For the quadratic terms, note that for the outer two terms:
(0) V (J) (
W
())
(i)
W
[0] (0) W (2) J1
()) (1)
W (0)
(0) V (5) V
W
())
W
[0] + (0) W ()) W
(0)
(191)
= (0) W (5) (i)
V
())
W
[0] + (0) W (1) (1
W
(1)
(
(0)
and for the inner two terms:
(0) W ()) ( V (5) (j)
W 00 (0) W
(1) (i) W (1) (j
V
10)
+ (0) V ())
(b) ( W [0] + (0) W W (J)
0)
(192)
=- (0) W (i) (i) V (5) (il W 10) + (0) V (1) (i)
W (J) (j)
W
(0)
We can then combine terms (along with the terms linear W) ) to see that:
21
Re (10)
(1) (1) W bi) G1 ( + +V)
N2
(193)
21o
Re (0) W
()) (1) 01 (W + V) 10))
rN2
40

---

Published as a conference paper at ICLR 2025
We can simplify the Hessian even further if we assume that " (1) by dropping terms that are w.h.p. subleading in g. In particular, consider the following terms:
rN ((0) V (t) W (j)
G1 V 10)}
(194)
(0)
18)
+ (i) w/)) (5)
VN-r GIV 10)
N
VN
Re (10) W ())
(b) W 100}
rN
(195)
N -
1 (0) W 8)
(8) V (1)
Gi w 10)
N
Re (10) W (1)
(5) (il
V 10)}
r. (N r) (0) w 1)) N -,,,, (1) 1b)
(196)
VN - G 10
VTN
Recall that the diagonal elements of 3- Wishatt matrix are 2 disttrribteed with 3r degrees of free- dom (see Appendix (A.4) and thus have exponential tails. In Lemma 43 proven in Appendix N we prove that the off-diagonal elements also have exponential tails. particular, all three of these terms are of the form RR R2 Rs. where 0 < K < 1 is some overall prefactor and the R. are random variables obeying tail bounds of the form:
IP [R, 2 1 + S K exp (-C min (L)
(197)
for some universal constants C, K > O. We thus have by the union bound:
37
P kR Rs 2 n3 +
P k max (Ri, R2, Ra)² 3 2 (st +1) 31
(198)
< 3 max (I
x Ri 2 xº + P nxt Rs 2 xt + - p 2 not +0)
(-Cmin (N= (N= - x 3²²)
3K exp
Taking into account the p² matrix elements via the union bound, these terms thus converge to 0 in Ky Fan metric at a rate Ω (R3) log < &p)). We can similarly use Lemma 41 to justify replacing W with low-rank perturbation W, which differs only in the removal of the first column of its Bartlett factor Lw. Altogether, we have that we can take up to an order
N (N r) log
P2N
& log
p² N
(N-))))
05 O
SO
(199)
N
No0i
error in Ky Fan metric:
2To TN2
Re (0) V (o) (1) W b)) (1 W
(200)
At this point it is instructive to consider this joint distribution in terms of the marginal distributions of elements the Bartlett decompositions W and V. Let Nw.., be the i.i.d. standard normally distributed random variables over F in the off-diagonal elements of Lw (with the convention i < j),
and similarly for Ny.ij and the Bartlett factor Ly of V. Furthermore, let 3 xw. be the i..dd. X- distributed random variables along the diagonal of Lw, and similarly for 3 Xv, and Ly.
We first condition on = 2. From Eq. (88 this is equivalent to conditioning on Xw.0 = / 2g
Conditioned on this event, the first derivative is distributed as:
L rN V To
20
r
=Xv. Re (iN (v..).
(201)
41

---

Published as a conference paper at ICLR 2025
Similarly, the second derivative conditioned on this event is distributed as:
21300
Re Nvo. i (i) W (J) Nw.o.)
(202)
rN2
l
Note that W is independent from Nw.o,1 exactly due to the removal of the first column of the Bartlett factor of W.
We will next simplify further using the fact that Xv.0 concentrates around its mean, in particular using the diiitribuution tail bound (for N r > 0, some universal constant 0, and bounded t):
P
1
(N r) 21 S 2eex (-OVN- rmnn (t,
(203)
(N - r)
which follows from Lemma
42
proved in Appendix
N Just accounted for cs, up to order
Ward (N r) to log
p² N2N
Nbboà log
pp An
N3 (NV
06
N
SO
N oo
(204)
error in Ky Fan metric,
2.
B
2x1l=
- 1) Re DeeiNN (ioodd
(205)
N
To
21 3o
B
4 =
-1) 2 Re Ni.. (i) W (5) Nw.os
(206)
N
Using Lemma 25
simplify this further as:
2130.
Bz
E1 =
Re (o..oo.N
(207)
N
lō
2130.
B2
Re Noo. (i) W (b) Nwas)
(208)
N²
To
We can simplify this even further by realizing that the distribution of Ny.o, is invariant under mul- tiplication by an overall phase, i.e., it is circularly symmetric. We can thus absorb the argument of Nwvo, into Ny.o, we simultancously conjugate W by unitary operators corresponding to this phase, so we need only consider the joint distribution of:
21,
Xi Re (iNvvo..
(209)
N
To
21
2130,
To (i) Re NV,,, W) 15),
(210)
N2
where here the X are i..dd x-distributed random variables with degrees of freedom given by:
X / [/|oool
(211)
Noting 3 Re (iN is distributed standard normal random variable proves Theorem 17 for sectors with = C, Hi, with accumulated emror in Lroy-Prokhorov distance:
p plog log (e- log (N- + p plog (L)
N log (N)
N log (N)
n' = Ti +og +200+220-100 + O
+1
+
Vi
NT.001
+h
(212)
We finally consider the second derivative conditioned both and all 2,1 0 (for 0). Note that the X distribution density is zero at O. In particular, this conditioning is equivalent to conditioning Re (iNv,,. 0. Furthermore, when F IH, recall that "t" could be any one
42

---

Published as a conference paper at ICLR 2025
In particular, this is conditioning on all nonreal components of Nv,0, equaling zero. Taking this all together yields:
of k due to Eq. (140)
==1== = 21,30
2 x Re (Nvo.) (i) Re W ())
(213)
N²
As the real parts of complex symplectic Wishart matrices are real Wishart with a factor of 3 more degrees of freedom and rescaled by a factor of 3-1, and similarly the real parts of standard complex symplectic Gaussian random variables are standard real Gaussian random variables rescaled by factor of (- ! we can more conveniently rewrite this as
L
210. N2
2 130
Nixi (i) ()).
(214)
where now W is a real Wishart matrix with Br degrees of freedom and the Ñ; are i.i.d. standard normal random variables. This proves Theorem 19 19for for sectors with F C, HI, with accumulated error in Lévy-Prokhorov distance:
p² Io'oog log log (N-+ p² log (1) log (W-
N log (N)
n
गा + 033 + 2014 a.s 06 =0
Vt
N
(215)
D.3.2 F= R
We now consider when F R. where we interested in the joint distribution of:
i = To
Tr (pxtox).
(216)
r
G, To
Tr (pxT
x (120) (2i + 1|- |2 +1 (21) XT o x):
(217)
rN
Las r22
To
TT (pXT
X ((2)) (2j +1 -11 2j 1) (2)l) XT,
(218)
X ((2)) (2i + 1| - 2: + 1) (i X x))
where now X a N (2p + 1) random matrix with i.i.d. standard Gaussian entries over As the commutator of symmetric matrices is antisymmetric, and taking note of the identity:
Tr (A = Tr ((C, (CC.AD B):
(219)
can rewrite this as:
i = To
Tr (pxTox).
(220)
l, Tr (pxT)
210
x 22i) X Ō x)
(221)
rN
44.5 rN2
21o
Tr (pxT
( (2j + 1|
x (126) (26 + 1| 126 + +1) (21)) X .6]]
(222)
Recall that
Ô
n)) (ml.
(223)
Let C be the complement of Ô, i.c.,
C= Σ pp) (pl
(224)
In particular, C' and Ô sum to the identity:
C Ō+Ô= In-
(225)
43

---

Published as a conference paper at ICLR 2025
Furthermore,
W = X ==XTXX
(226)
(227)
are each distart-distriuueed matrices; the former with r degrees of freedom, and the latter with - The independence of W and V can be checked from the associated Bartlett decompositions. We thus have that the distribution we are interested in is the joint distribution of:
i = T (0)
W 10),
(228)
210
4 (0) (X X (2)) (2i + 1 W W 22)) (2i + 1| 10)
rN
(229)
210
(0) W ((21) (2) + - |2 - 1) (2 V 10),
rN
and:
21
(0)
X [X (2)) (2) + 1 X X ((2)) (2) + 11 1 + 1> (2il) X O]] x 10)
N2
2To
(0)
x (2)) (2) + X (22)) (2) + 1| - |2 + 1) (1 X (0)
rN2
210
(0)
X ((26) (2i + 1| - 22 + 1) (2il) X (2)) (2j + 1| X 10)
N2
210
(0)
X ((21) -21+11 1| - 2 1) (2)) X 2)) (2) + 1| x 10)
N2
210
(0)
X X (2j) (2) + 1| X (2) (2) + 1| 22 + 1 (21) X X 10)
(230)
N2
2To
(0)
(W +V) (2) + ( V) (22) (2i + 1| 121 + 1) (2 W (0)
r22
210
(0)
W ((21) (21 1 1) (2 W V) (2) (2j (W V)(0)
r22
210
(0) (W + V) ((26) (2) 1 | + 1) (2 W 2)) (2) (W + V) (0)
T22
210
[0) (W + V) (2)) (2j + W ((2)) (2i 1 - 2 + 1 (2 (W + V) 10)
N2
We can simplify the expression for the second derivative by grouping terms by their order in W. Note first that all terms cubic in matrix elements of W sum to zero For the quadratic terms, note that for the outer two terms:
+ (0) V (2)) (2j + W ((2) (26 + 1| [2 + 1) (2il) W (0)
+ (0) W (25) (2j + V ((2)) (2i+1] 1| [2i 1) (2il) W (0)
(0) V (2j) (2) 1| W ((2)) (2i+1) |2i+ (2il) W (0)
(231)
(0) W (2)) (2j W ([21) (2i 11
11) (2il) V (0)
=+ (0) W (2)) (2j ((2i) (2i 1|
[2i 1 1) (2il) W [0)
(0) W (2)) 1| W ([21) (2i+1 + 1|
|2i + 1) (28))
V [0).
and for the inner two terms:
+ (0) W (22) (2i 22 + 1) (21) V (2) (2) + W 10)
+ (0) W (21) +1| - (2 W [2j) V (0)
1)
(0) V (124) 11 1) (2i)) W [2j) [
W (0)
(232)
(0) W ((2)) (1 - +1] (24) W (2j) (2j+1| (0)
=+ (0) W ((2)) (2i 1+11-22++1 1) (24) V [2j) (2j+1) W (0)
(0) V ((2i) (2 2i + 1) (2il) W [2)) (2j+1 W (0)
44

---

Published as a conference paper at ICLR 2025
We can, in turn, combine these terms to obtain:
+ (0) W (2) (2) + ((2)) (2i + 2 + (2i)) W (0)
(0) W (2)) (2j+11
+ 1|
W ((2)) (2i + 1|
V 10)
+ (0) W ((2i) (2i 1| 2 + 1) (2i)) V (2] W (0)
(0) V (121) +2 [2i +1 (2il) W (2)++|1 W (0)
+ (0) W (2)) (2j+ V (12:)
-1++1 W (0)
(233)
(0) W (2)) (2j+11 W (2) ----1|-(2++)) 12 V (0)
(0) W 2j (2)]
[-c1|-1ii+0 - W [0)
V
+ (0) W 22 1) W (12)) (2 (2)))
=+ (0) W ((2) - (2)) V ((2)) (2 1
2 1) (2il) W (0]
(0) W ((2)) 22j+1|- [))+ (2))) W ((2)) (2i 1| [2 1) (2 V (0).
We can then combine terms with those linear in W to see that:
210
(0) W ((2:) (2i + 1| 22 + 1) (2 ((2)) (2j + 22j 1) (2))) (W + V) (0)
r22
2
rN2 (0) V ((22) (2i + 1| |2 + 1> (2 W ((2)) (2) + 1| 22j + 1) (2))) (W + V) 0).
(234)
We now use two lemmas proved in Appendix N that will simplify the form of our answer by al- lowing to justify the dropping of terms that are subleading w.h.p. In particular, we use Lemma 43
followed by Lemma [4] as we did in Eqs. (94). (195). and (96).. By the same logic, we have that we can take up an order
N33 (N - r) log
pNN
No log
pNNto3 No
(N))3
05 0
50
(235)
N
N N
error in Ky Fan metric:
21ō
eas r22 (0) V ((2)) (2i + 1| |2i + 11 (2:)) W (12)) (2) + +1| [2j 1) (25l) W (0)
(236)
where W is as W, but with the first column of its Bartlett factor Lw removed.
At this point is instructive to consider this joint distribution in terms of the marginal distributions of elements in the Bartlett decompositions of W and V. Let Nw.a, be the i.i.d. standard normally distributed random variables over & in the off-diagonal elements of Lw (with the convention < ÿ), and similarly for Nv.i,j and the Bartlett factor Lv of V. Furthermore, let XW; be the i.i.d. x- distributed random variables along the diagonal of Lw, and similarly for XV,; and Ly.
We first condition on é = 2. From Eq. (88) this is equivalent to conditioning on Xw.0 = V T Conditioned on this event, the first derivative is distributed as:
21
4.1==
=xvo
10x00,2411Nv.... Nw.o.2.N..0.2.5.
(237)
rN
To
Similarly, the second derivative conditioned on this event is distributed as:
21o
Gagl== rN2
2XV.0
1
(238)
X (Nv,0,2441 (2il Nv,0,2 (2i + 1)) W ((2)) Nw.0,2541 [2j + Nw.02))
Note that W is independent from Nw.o,; exactly due to the removal of the first column of the Bartlett factor of W.
We will next simplify further using the fact that Xv.0 concentrates around its mean, in particular using the x-distrbbution tail bound (for N- r > O, some universal constant C > 0, and bounded I):
1 (N - r)a x V B(N - r) 2 < 2exp (-CVN -rrin (1, (1,
PP
(239)
45

---

Published as a conference paper at ICLR 2025
which follows from Lemma 42
proved in Appendix N
Just accounted for as, up to order
Narr (N r) la log
N
Noo log
p'N Mod
ag
0
(240)
N
N N+
emror in Ky Fan metric,
21
N
Ex1=
V
-
N Nw.o.2N..0221).
(241)
N
To
210
(N
[As1= N2
V
1ō
(242)
(Nv24,244 (2il - Nv,22 (2i + 11) W ((2)) Nw [2j Nw.u22)
Using Lemma 25
can simplify this further as:
210.
2,1 =
(Nw.0.24411 Nv.0.21 - Nw.02.N....511)
(243)
N
To
21a.
[xJ]2 = N2
(Nv.02241 (26) Nv.022 (2i + 11) W ((2)) Nw.022411 - 12] + +1) Nw.o,25)
(244)
We can simplify this even further by realizing that the joint distribution of (Nvv,022. 22 is invariant under orthogonal transformations, i.c., it is circularly symmetric. We can thus simultane- ously transform (Ny Nv.0,21+1) 1) and conjugate W by orthogonal operators such that we need only consider the joint distribution of:
2Ta.
X.NVv,0.2+1.
(245)
N
lo
210.
2
Xi (Nv,0,2 (2il - Nv.0,241 (2i + 1)) W (2)),
(246)
N²
where here the X are i.i.d. d-dittrbbuted random variables with degrees of freedom given by:
x== V N2 W,o.2: + N W.22+111
(247)
This proves Theorem 17 for sectors with F = R, with accumulated error in tance:
dis-
p plog (1) (plog log (2)) *log
N log (N)
N log (N) +
n' n. +a3 204 06 = O
(' (N-te-1) +
Vt +
N1.001
f
(248)
We finally consider the second derivative conditioned on both - and all bz 0 (for > 0). Note that the x distribution density is zero at O. In particular, this conditioning is equivalent to conditioning on Nv.0,24+1 0, yielding:
2To. N²
V To
Nv.0,2.3 (2i) W (2)
(249)
As submatrices of Wishart matrices are also Wishart, can more conveniently rewrite this by taking W to be
210.
L
To Nos (i) Wb)
(250)
N
where the Ñ; are i.i.d. standard normal random variables. This proves Theorem 19 for sectors with R. with accumulated enror Lévy Prokhorov distance:
-0005500 π = क. +a3 + 204 +as - -0s = O p² log log (W-
p2 log (t)
Vt + N log (N) ) (251)
log (N-1e-)
N
46

---

Published as a conference paper at ICLR 2025
E QUANTUM ALGORITHM FOR DETERMINING ASYMPTOTIC TRAINABILITY
We elaborate here on a claim made in Sec. that one can efficiently estimate Tr (109)²) and
Tr ((a))) on quantum computer and thus, by Definition 7, determine whether given quantum neural network architecture is asymptotically trainable or not. We assume as input the Jordan algebra A = θ Aa associated with the model, given as orthonormal bases {Ba.ih; spanning each A.. (09)²) and Tra ((0*))) in the defining representation of Aa can then also be evaluated using Eq. 11
Note that the corresponding traces Tra
(+)- = la Tra (:);
(252)
in particular, Ta can be calculated from the ratio of traces of any Ba in the defining representation of H (C) Aa and that of Aa-
We begin with Tr (10)))) is immediate from the orthonormality of the Ba that:
Tr (109)2) - Σ - Tr(:..:))
(253)
in particular, Tr (())))) is calculable via, for instance, phase estimation (Niclsen & Chuang 2010a) of the Ba
This leaves only Tr (107)²). If O were efficiently preparable, valid quantum state, one could
immediately calculate Tr (Oaa))) in the same way Tr ((D9)²). In the more general case where, for instance. O is given via its Pauli decomposition:
0 P.
(254)
1=1
one can proceed similarly via block encoding using the linear combination of unitaries (LCU)
subroutine (Childs & Wiebe 2012).
To outline this procedure, we assume O is an observable on n qubits. Let:
g
(0)
P. 2 (I2 Z)
(255)
2
i=1
where N is the NN x N identity matrix and Z the 2 x 2 single-qubit Pauli Z matrix, and consider the (n 1)-qubit quantum state:
I2-
Wp = 2"
@ 10) (0)
(256)
We have by construction that:
10) (0) = f (0) wof (0).
(257)
Thus,
Tr (10m))
Σ Tr (Bo,,
(258)
= -Tr (Ba.)) (0) wof (0))
Of course, f (O) is not necessarily unitary, so naive application of f (0) is impossible on quantum computer. However, one can proceed using the standard LCU subroutine (Childs & Wiebe 2012). Assuming O is normalized such that its root mean square eigenvalue is θ (1),
Tr (f (O) wof (O)) 2 Tr (02) +1).
(259)
Following the protocol described by Theorem of Chakraborty 2024). then, each
Tr (Ba.)² (0) wof (0)) can be estimated to an additive error e using one ancilla qubit and samples through an LCU block encoding.
47

---

Published as a conference paper at ICLR 2025
F OPEN QUESTIONS
As our results are formal, there is room for violation of their conclusions in the settings where their assumptions do not hold. One example this the local minima distribution of quantum neural networks discussed in Appendix C.3 The results given there assume that the eigenvalue spectrum of the objective observable is concentrated as is typical of low-weight fermionic (Feng et al.) 2019) and local spin Hamiltonians (Erdös & Schröder 2014). However, the behavior of the local minima density differs when the spectrum is not concentrated, as is typical of observables drawn from the Gaussian unitary ensemble (GUE) and nonlocal spin systems. Observables from this latter class known to be "quantumly easy" in the sense that phase estimation performed on the maximally mixed state efficiently prepares the ground state (Chen et al. 2024). Based on this intuition these nonlocal systems may also yield local minima distributions for quantum neural networks more amenable variational optimization even shallow depth. If this true, this may give natural class of optimization problems which are easy solve given access to a quantum computer.
Furthermore, we nowhere discuss the impact of noise: we assume all operations implemented by the quantum neural network are unitary, not general quantum channels. It is already known that noise can affect the presence of barren plateaus (Wang et al., 202I) as well as local minima (Li & Hernandez 2024 Mele et al. (0224) in variational loss landscapes. Heuristically the former can be understood as an additional channel mixing the variational ansatz over Hilbert space, and the latter noise effectively limiting the number of parameters influencing the variational ansatz and thus always keeping the network in the underparameterized regime. We hope in the future to make this heuristic understanding rigorous.
G REDUCTION OF SPIN FACTOR SECTORS TO REAL SYMMETRIC SECTORS
i.e. We elaborate here on the claim made in Appendix B.1 that the where Aa = LN. -i.., the spin factor sectors-are conceptually equivalent to the case A, H.. -1 (R). To see this, consider a parameterization of 7 (0) in the spin factor case:
P II
( 90
o exp (0,A
(260)
where ofº E SO (N 1) and A E so (N 1). As stated in Appendix B.1 of acts trivially on the first component in the defining representation. Because of this (assuming the loss shifted by constant to always be nonnegative). the associated fxr (0; p) is just equal to the square root of an instance of the H.N. (R) case with initial state:
= TT 2 p
(261)
and objective observable:
Ō = 0
Ø 0
(262)
H REDUCTION OF PREVIOUSLY STUDIED ANSATZ CLASSES TO JORDAN ALGEBRA-SUPPORTEE ANSATZES
We here show that Jordan algebra-supported ansatzes (JASAs) are reduced to by the algebraically defined classes of variational ansatzes previously introduced in the literature, with primary focus on the Lie algebra-supporter ansatzes (LASAs) (Fontana et al., 2024). A reduction in the other direction-i.e. given Jordan algebra-defined ansatz, define an equivalent LASA-is not possible. As simple example, consider the case when the objective observable O is real symmetric matrix and the variational ansatz is PT symmetric (i.e., real) but has no other symmetries. This fits our framework with Jordan algebra A ~ HN (IR) and automorphism group given by the action of conjugation by SO (N). However, so (N) in the defining representation consists of antisymmetric matrices, this is not LASA. We also give reduction from the variational matchgate circuits of Diaz et al. (023) to our Jordan algebraic framework, and will later demonstrate that such a reduction to LASAS not possible.
48

---

Published as a conference paper at ICLR 2025
We first discuss how JASAs are reduced to by LASAs. The loss function of a LASA is of the form:
p0: (p. ( O),
(263)
where O belongs to some dynamical Lie algebra g. T (0) is the adjoint action of some element of the compact Lie group G associated with g. and p arbitrary. As demonstrated in Fontana al. (2024), only the projection of p onto the contributes to the loss, so we consider when E g WLOG.
G is a compact Lie group. Thus, up to an Abelian sector that does not contribute to the loss (as O g). G can be written as the direct sum of simple compact Lic groups. By the well-known classification of simple compact Lie groups these are either the classical so (N), su (N). sp (NV, or one of the exceptional Lie groups. The exceptional Lie groups are of fixed dimension-iee there is no sense of an asymptotic limit- so we consider here the classical cases only.
We begin with the case z su (N), with defining representation do over C. It is easy to see that i d (O) and dæ (p) are the defining representation of the Jordan algebra H (C), with identical trace form. Such a simple component is thus equivalent to JASA with simple subalgebra isomor- phic to HN (C).
We now consider when g = su (N), with defining representation dø over H. As the loss is real- valued we have the expansion in this sector:
e (0: p) x Tr Re (idø (p)) do (U (0)) Re (idø (0)) de (U (())))
+ Tr (Re dj (p)) dø (U (0)) Re (j dø (0)) do (U (0))
(264)
+ Tr RRe do (p)) dø (U (0)) Re (k (0)} dø (U (0))')
Each of these real parts is Hermitian and can be written as elements of the defining representation of HV (H), reducing this simple component that of JASA.
We finally consider when g 2 so (NV There are more subtleties compared with the other classical Lie algebras as, in the defining representation over R, there is simple relation between Hermitian and anti-Hermitian operators. To make the following concrete, we here consider the representation of (N) defined by Majorana operators Vi with generators:
My 2 [n, 77l,
(265)
i.e. polynomials quadratic in Majorana operators. These are the generators of the spin algebra spin associated with the vector space spanned by the "% and thus is isomorphic to so (N). Furthermore, the gates generated by the Mij are just matchgates. The loss function variance of variational match- gate circuits was studied in detail in Diaz et al. (023)). The authors there also consider when O & g. making this an example of a barren plateau result beyond the LASA formalism.
We now demonstrate how this setting fits into the Jordan algebra picture. Let be the Jordan algebra of Majorana fermions, with Jordan multiplication given by half the anticommutator. As in the Lie algebra picture, the dimension of this algebra grows exponentially with n; similarly, the Jordan al- gebra generated by k-body (2 < k < n) Majorana terms has dimension growing exponentially with n, as does the generated algebra when acting on these k-body terms via conjugation by a matchgate. However, though these operators are not closed as an algebra under matchgate conjugation, they are closed as a vector space. Furthermore, the trace form of a k-body term e and a k k'-body term a/ is always zero when # k' This allows us to instead consider O as a member of a different Jordan algebra A', with the same underlying vector space as A and new Jordan multiplication operation defined as:
1
if + = V' if o/ 1,
(266)
if =1,
0
otherwise.
Note that this new multiplication operation still satisfies the Jordan identity:
(1b
(b)
v
(267)
(2) 4))
49

---

Published as a conference paper at ICLR 2025
It also preserves the canonical trace form, Tr (To (+b o v'") 2n 80.0" = Tr (T. (+b xf'))
(268)
Here, L. is the linear transformation associated with the Jordan multiplication ::
L. (u) u
(269)
and similarly for Tog The loss function in this sector thus a JASA under the Jordan algebra A', and thus both general matchgate ansatzes or the LASA setting with = so (N) reduce to JASAs.
Interestingly, one can show that the matchgate setting cannot generally be described by a LASA while maintaining the trace form, even when the Lie bracket is redefined. To see this, we consider adjoint endomorphism
(d (u) [u, 2]
(270)
associated with the Lie bracket [+ -]- The canonical trace form for Lie algebras is the Killing form, given by:
K (u, v) Tr (addaa (u) ad (v))
(271)
Note that (as (..) (u) is antisymmetric):
K (u, u) = ---ldadd.,) (w)lF
(272)
For u a Majorana monomial under the usual Lie algebra of Majoranas, this is just (minus) the number of terms anticommuting with the monomial. For degree-1 monomial this will grow exponentially in the number of Majoranas n. If instead one considered defining a new Lie bracket I that is zero when mapping from k-degree Majorana monomials to those of different degree, K (+r-) would always be zero on degree-1 Majorana monomials-in particular, the trace form not preserved when modifying the Lie bracket.
I REDUCTION OF LOW-PURITY INPUTS TO MAXIMALLY MIXED INPUTS
We elaborate here on the claim made in Appendix B.2 that the input state projected into simple sector A. has sufficiently low purity, the loss contribution from this sector does not contribute in any meaningful way. This is true even when the loss is rescaled by some N < O (No.99) as we consider elsewhere. Intuitively this due to pa being close maximally mixed and thus the contribution to the loss in this algebraic sector is effectively constant. We formally state this claim as the following lemma.
Lemma 29 (Low purity inputs approximately trivial). Let J, be as in Theorem 13 For
E R. such that for some a::
Tr (189)² S NN.999
(273)
we have for sufficiently large c-!, ,, that:
A/2
Var We (0:p)) 50
(z)==00(1
(274)
N1
for all N 50 (N.
Proof. Assume Tr (p") = WLOG by otherwise absorbing a factor of Tr (p*) into O Consider the decomposition into eigenspaces:
p σ + Ta,
(275)
where the nonzero eigenvalues of " are the largest N0.909 dure
eigenvalues of p". Iterate this proce-
N
Ma
(276)
No.999
total times to yield the decomposition:
Ma -1
pa
σ +
(277)
50

---

Published as a conference paper at ICLR 2025
The standard deviation of sum of random variables is upper bounded by the sums of the standard deviations. Furthermore, from Eq. (277) we have that
Tr (109))
2 Tr ((09)²).
(278)
We can thus upper bound the standard deviation of the loss by applying Corollary 20 to the terms in Eq. (277):
TF ((om)?) Tr ((om))
Var (Nee (0: S
(
dimR (ga)
Tr (10m)²) Tr (pa)
(279)
S S M2
+ +0(T)
dims (0x)
A²2
5
+O(T),
N1.997
yielding the final result.
J c-APPROXIMATE 2-DESIGNS SUFFICE FOR THE G;
We here argue that one need only assume that the G, form e-approximate 2-designs not large-t t- designs-if takes the overall normalization A to be at most (N3) and if p grows sufficiently slowly with the Na. More concretely, instead of taking Assumption 15 we may take the following assumption:
Assumption 30 (Scaling of parameter space with ansatz moments). The number of trained param- eters p satisfies
log I
1² so min
(280)
log log WV-IN2
where the 9 are i.i.d. e-approximate 2-designs over To Auti (Aa), where
(281)
To achieve this we will rely on tools from random matrix theory to bound the error of various mixed cumulants of expressions under such change. In particular, will use the existence of so- called limit distributions of all orders.
Definition 31 (Limit distribution of all orders (Collins al. (007 A sequence of sets of N N random matrices {A,) has limit distribution of all orders if there exist functions og such that the cumulants K& obey as N + 0o: lim l1l,, -2 (Tr (Ai,..., As).,,,,)))..... As)),... ..... Tr. (pr (Ai...., ..))) N
(282)
aa (p1 (AI..... As),.. ......P.. (A (A,...., A.))
for fixed polynomials p.. Here, r |
Simple classes of distributions which satisfy this property include constant matrices of bounded operator norm, Gaussian random matrices, Wishart random matrices, and Haar random unitaries. We here will be interested in a weaker notion of the existence of a limit distribution of all orders, requiring only that higher-order moments vanish sufficiently quickly and not necessarily that they have well-defined limits.
Definition 32 (Weak limit distribution all orders), A sequence of sets of {A has a weak limit distribution of all orders the cumulants K& obey as N + oc:
random matrices
(Tr (pi (Ai...,, A.)) ..... Tr (wr (Ai,... A.))) = O 0 (22-1A1l)
(283)
for all fixed polynomials pj. Here, r |
As in Assumption 15 this be weakened to bound p if one is only interested in the gradient.
51

---

Published as a conference paper at ICLR 2025
Products of matrices with limit distributions of all orders F R. C,H considered in Mingo & Popal (2013); Collins et al. (2007); Redelmeier (021) respectively, allowing us to prove the following lemma.
Lemma 33 (Existence of weak limit distributions of all orders), Let Li i E [d] be multilinear functions of the form:
= 'I' (MhOhh). where h is Haar random and independent from the M,. Assume all M; have weak limit distributions of all orders. For any cumulant KR. (Lis..... Li.) with ||A=|], 2 3,
(284)
Kik (Lig..... Li,) = O (N² (N2-1Ai)) [kll1
(285)
Proof. Following:
1. Lemma 5.12 of Redelmeier (0122 when R;
2. Remark 4.8 Collins et al. 2007 when F= C
3. and Corollary 3.2 of Redelmeier (2021) when F HH..
the product of matrix from unitarily-invariant ensemble with any other independent matrix with a weak limit distribution of all orders has a weak limit distribution of all orders. This implies that the M have weak limit distribution of all orders. The vanishing of higher-order cumulants as in Eq. 2855 then follows from Definition 32
We thus have the following lemma.
Lemma 34 (Weak convergence Haar random gi). Assume Assumption 30 uniformly bounded multilinear functions of the form:
Let Lis i [d] be
L Tr phohl II 9,
(286)
JEI,
where I; is some index set associated with i of constant cardinality, h is Haar random, 9j 9j, and the M; are fixed with bounded trace norm, and d =
log (N-IN3 log log (N (N-1 Let L, be the same The joint distribution of N NL., differs
multilinear functions, where now the 9j are Haar random. from the joint distribution of N Li by an error at most
dloo (N- -N
122 =
(287)
(N-IN3)
0.99
in Lévy Prokhorv metric for any N (N.).
Proof. The gj M,9, have a limit distribution all orders due to the M, being fixed. Thus, itera- tively one can show the trace arguments of the L; have weak limit distributions of all orders by the results cited the proof of Lemma 33 Incorporating the bounded trace norm of the M, and normalization A gives an error for the cumulants of order k 2:
غ O (N* 0 ((NN~))))
(288)
This also bounds the 1, cases whenever
e e<o (VO
(289)
which is true by Assumption 30 in the proof of Lemma 26
The result then follows from Corollary 40 where is subleading
52

---

Published as a conference paper at ICLR 2025
K EQUICONTINUITY OF PROBABILITY DENSITIES
In Appendix D we prove that the joint distribution the loss function and its first two derivatives for certain random class of variational quantum loss functions converges in distribution to fairly simple expression involving Wishart-distributed random matrices. However, Corollaries [14and [8] stated in terms of the probability densities of the loss and first derivative, and weak convergence does not necessarily imply pointwise convergence in densities. However, it is known that this is the case when the sequence of densities are equicontinuous and bounded (Boos) 1985). Whether or not this is true depends on the details of the distributions g. H. As an example, we here show this is true for the loss when G, are Haar random, under reasonable assumptions (which we conjecture are not necessary) on the spectrum of the objective observable O projected into any simple component.
Lemma 35 (Equicontinuity of loss density with Haar random ansatz). Consider sequence of O as N + oo with eigenvalues:
0= S 02 ... SNN ON 1
(290)
with mean eigenvalue and TY(O")N
Tr))
N (1). For E RN- define:
Ev v ō (w 2 w)
(291)
R v /wu w
(292)
where ō is the vector (o2...., 11 and @ denotes the Hadamard product. Let D be the domain of w satisfying:
Es S So+ 1x E -R2
(293)
Assume
dw (1- (1-(O+N
P.re (z)
E
(294)
V N (o+ SN Dr
locally Lipschitz and bounded at any x as N - co, with Lipschitz constants independent from N. Then the density of vrl is bounded by constant independent of N and is equicontinuous sequence in N.
Proof. By assumption r θ (N) so we consider the density of / WLOG. Let {lot) be the eigenvectors of O associated with the eigenvalues 0g. Let:
N
Iu) =
u; lo,)
(295)
1=1
and let:
S Su) = Σ o8 |u,|
(296)
Let P.rTe (x) be the density of VNÉ= = VNo+ +x. x. We have that:
PVNe S f | du 8 (VN(S(u) -0)- (v))
(297)
Here, SaN-1 is the surface area of the (BN 1)-sphere:
2TT 8NV
SpN-1
(298)
r AN
We can integrate out the phases of the u; and note that the delta function has no dependence on Ui
since 01 = 0. Therefore,
PuNe (z) SW-12N
/
dud d (VN(S
(S σ) z).
(299)
$1
ū20
53

---

Published as a conference paper at ICLR 2025
Here, ū No with the 0r = 0 component projected out.
N-1 has indices in .,..., N such that ü; |ug]- We will similarly use ö to denote
We now proceed to show equicontinuity and uniform boundedness of ps (a) by showing that the derivative of Py (z) is bounded everywhere by some N-independent constant. By symmetry we restrict to the components of uN 2 0, introducing
overall factor of 2 We also introduce new coordinates w2,...;; WN, where w; = üg for S i S N and wN = (ū). The constraint " 2 O becomes in these coordinates:
N 1
0, < WN.
(300)
Only the final row of the resulting Jacobian is nontrivial and is equal to:
JN 2 u
(301)
where @ denotes the Hadamard product. This gives Jacobian determinant of (recalling that oN 1):
N
det (J) 2ūN 2, WN
Σ
o,w7
(302)
1=2
Putting everything together yields density:
(VN VWNNNN (WN - 0) -
/ dw
Pune (x) 2 22
(303)
VuN EN-1 owwo
where D is the domain of positive wy obeying the normalization constraint of Eq. (300 well
N --
UN +
Σ
(1 - o1) o;) w S 1.
(304)
Defining:
N-1
E
0 7
(305)
N
R2
(306)
10
1=2
and rescaling we can rewrite this as:
P.NE (x) VNSN-1 /
N
+1
δ [WN /No x
dw
VN dwn
(307)
JN-
VN WN E2
where -22 is the unit (N - ball. Integrating out the delta function then yields:
(x)
1
S
dw 1 N
EZG
(308)
D=
N N SN-1
where D2 is the domain of úp satisfying:
E 5 1 E2 R2
(309)
By assumption this is locally Lipschitz and bounded as N + 2o with Lipschitz constants indepen- dent from N, and thus is equicontinuous and bounded.
54

---

Published as a conference paper at ICLR 2025
We now discuss the main assumption of Lemma 35 in more detail, particularly the local Lipschitz continuity of Eq. (294). Note that the defining equations of D= imply that all w/ Σ 1, ie.. the domain of integration is always of O (1) volume. This holds true even if some o vanish with N; it is apparent that these terms do not contribute to the integrand at leading order O, and only the Q (1) eigenvalues contribute. Because of this, we will perform an example calculation of the Lipschitz constants of Eq. (294) when all o; = ō 2 for + N, though due to this observation similar results hold for all reasonable spectrums.
For such O,
1 2
Reg
(310)
so
v2
dw 1 (1-(1+20)) 2N (1++2N+x) 1
PuNe (x) =
N (1+ -2x -N-1
D2
V28N-3
1-22 1|01
dr N-3 1 (1+2N 2N 22=22 (
12)
N (1 + 22-22222 tx,0 SN-1
SN-3
1-2N [x|
drr 2 -2 (1- 1-(11 (1+ 2N 2222222 22
1111 r)
-3
V 2N (1+ 2N tx) SN
3/4 SN-3
11-N
2N
|0o)
1 - 2N |20
drr
1 + 2N
V 2N (1+2N- ++ 8N-1
11-2N-3 |x|]
r
-
1- 11-2N-1 2N |x|
drr -2
1 2N
TV 2N (1+ 2N 2x)
( 1)
(311)
This integral is just the integral definition of the hypergeometric function 2F yielding:
(1 - 2N |x|]
1 N
N 1 1 2N 2 |o||
PVNe (x)
F1
1;
(312)
2 2
2 1+ 2N 3x
T 2N (1+2N 1x)
We now consider the large-N asymptotics of the derivative of this expression. By Gauss's hyperge- ometric theorem, for + O (N-
N - 1; N 2 22
(1+0
r ( ( (())))(())
- (VN)
F1
(313)
2 2
r
Similarly, we have by the derivative rule for the hypergeometric function that:
1 N N 2
3 N N
= -N(N).
=F'
):²)
2
- (1 - 22-22-1) Fi
2 2
+ Liz =
(314)
Taking into account the N factors scaling thus yields (where defined):
V (x: = (1).
(315)
In particular, P.rze (z) is locally Lipschitz with Lipschitz constants independent from N. These 0 thus satisfy the assumptions of Lemma 35 claimed.
55

---

Published as a conference paper at ICLR 2025
L BOUNDING LARGE DEVIATIONS OF THE HESSIAN DETERMINANT
In calculating the density of local minima in Appendix C,3 we require analyzing the asymptotics of:
IE [det (D:) O}]-
(316)
where D2 is as in Eq. (124), i.e., it is a Wishart-distributed random matrix normalized by its degrees of freedom parameter. As the determinant is exponentially sensitive to its argument, exponentially unlikely deviations in the convergence of the spectrum of D2 to its asymptotic value can, in prin- ciple, cause this expectation to not converge to the determinant of the (almost sure) asymptotic spectral measure of D.. The same is true of the minimum eigenvalue of D.. We here show that these deviations do not asymptotically contribute to Eq. (316). We will here be brief as this is not the main focus of our work, and instead refer the interested reader to Anschuetz (2022) for a more detailed account of very similar arguments. In the following we consider only the case when 7 S as otherwise D is never full-rank and Eq. (166 is identically zero.
We begin with the empirical spectral measure of Dz. The empirical spectral measures of W satisfies a large deviations principle at speed p² with good rate function maximized by the Marčenko- distribution duuup (A) (Hiai & Petz 1998). As the determinant is only sensitive at a speed p. i.e.
det (D2) = exp (p) f dypup (2) (A)),
(317)
by Varadhan's 's lemma (Dembo & Zeitouni 2010) the expected determinant converges to the deter- minant of our asymptotic expression for D..
We now consider the smallest eigenvalue of D.. For fluctuations below its asymptotic value this satisfics a large deviations principle with good rate function at a speed p. where this rate function is maximized the infimum of the support of the Marčenko-Pastuu distribution; for fluctuations above its asymptotic value it satisfies a large deviations principle at a speed p2 (Katzav & Pérez Castillo, 2010). By Varadhan's lemma, then, only fluctuations below the asymptotic value potentially con- tribute, but as these fluctuations only decrease the value of the argument of Eq. (316) they do not contribute asymptotically (Dembo & Zeitouni 2010). These together imply that Eq. (316) converges as claimed in Appendix C.3
M LEMMAS ON CONVERGENCE IN DISTRIBUTION
We here prove a few helper lemmas that will allow us to obtain quantitative error bounds on the rate of convergence of two sequences of distributions given bounds on the differences of their moments or cumulants. We will achieve this by bounding the Lévy-Prokhorov distance between the two distributions, which metricizes weak convergence. We first restate a result of Berkes & Philipp (1979) which bounds this distance, with a slight modification due to our use of the infinity norm rather than the Euclidean norm (see Appendix A.5))
Theorem 36 (Slightly modified version of Lemma 2.2. Berkes & Philipp 1997)). Let p ,9 (æ) be two distributions on Rd with characteristic functions @ (()), 7 (u), respectively. Assume there exists C such that S1 (x) S μ. There exists a universal constant K such that for all T 2 max (2C, Kd), the Prokhorv distance between p and q is bounded by:
T π
d
(w) - 7 (u)) du + 16 (T))
æ (p. 9) <
+ p.
(318)
T
ST
Motivated by this, we prove a general bound on the error of the characteristic functions given errors in the moments.
Lemma 37 (Bound on characteristic functions from moments). Let (æ) q (x) be two probability densities on Rd with characteristic functions & (u) : (u). Assume each moment of order
of
p differs from that of q by an additive error at most > 0, and assume all moments of order k > t
k are bounded by (CvR) for some k-independent C 0. Then, for all u such that ||ul|1 S 2cC
YFT
10 (u) (uu) S &eep (1|u011)) + exp2 (1 1).
(319)
56

---

Published as a conference paper at ICLR 2025
Proof. We have by the definition of the characteristic function and the triangle inequality that:
(iu
(iuy)
(w))) - - ( S Σ
II
2
Cillll
(320)
ky!
kgg
KyDENg j=1
|
2++1
The first term has the upper bound:
d
(
Σ
II
S eexp (1/u)li).
(321)
kz!
j=1
For the second term, we have the upper bound (for ||aa|1
VI+I
e0
Σ
|||||
[|kil
2
d
U1y
kj
liveCuli
kig
Vk!
20t1
==1
12
eCu
(322)
vl+ 1
12
141
eC VI+I
||ze|1
|/20|1
VzTT
When ||2||
<
VFI 200
this gives the bound:
(iuy)
eC
1+1
2
Cllkll ||]]]
|12|||
I exp2 (1-)).
(323)
ky!
Vt+I
|AII,
24+1
Combining these bounds yields the final result.
This characteristic function bound combined with Theorem 36 gives the following corollary.
Corollary 38 (Bound on Lévy-Prokhorov metric from moments). Let px (æ), qN (œ) be two se- quences of distributions on IR with characteristic functions ON (u) 7N (u), respectively. Assume each moment of order k S pN differs from that of by an (N dependden)) additive error at most
and assume all moments of order k. (for N N-dependent)
bounded by (CvT)) for
some (k, N) -indepeend constant C. Assume well that [[0] min 22 in(e-1), VET
P (1x)
2o0d
ju for some (N (e-ependent) The Lévy-Prokhoron distance between PN: 9N is bounded by:
(d log log (6-1) dloog (1) log (E-1)
π (pN.9N)
+ 41
(324)
VE
Proof. Let T' min
0.99 d
In 16-
V+1 BeCd
From Theorem 36
ΦN (u) YN (w)l + O dl log (T)
(pN.ggN) (N) s (r)d)
sup
+ +0 (p).
(325)
||ll
T
From Lemma 37
sup
IØN (u) - YN (u) 0.01 + exp2 (1 -t).
(326)
[ull b0
log(c-)
)) as
The convergence rate given in Eq. 324 is trivial when d = Q min
log log(e 1j log(())
Ing(c-1) --,6,, + co. When instead d min log Logíe 1y' log(t]
v
2d exp (0 (min (log (e-1), vo
(327)
The second term in Eq. 325 thus dominates in this setting, implying the final bound.
n
57

---

Corollary 40 (Bound Lévy- Swyy-Prokhorov metric from cumulants). Let pN (æ)) IN (a:) be quences of distributions on IRd with characteristic functions ON (u)) .N (u), respectively. Assume each cumulant of order of pn differs from that of qN by an additive ervor of at e > O. Finally, assume that Published as a conference paper at ICLR 2025
We also prove bound the error of characteristic functions given bounds the cumulants.
Lemma 39 (Bound on characteristic functions from cumulants). Let (a)) .9 (x) be two probability
densities on Rd with characteristic functions & (u) (u). Assume each cumulant of order of
differs from that of q by an additive enror of at most e 0. Then
j (u) 7 (u)) & max (exp (exp (e || 1) 1, exp (1 (1- -epp (E (ellul)))))
(328)
Proof. We have that:
(u) 7 (u) - 1 (u)) exp (In (7 (u)) - In (6
(329)
= =|1 - cxp (In (u)) (ó (x))))
By the definition of the cumulant generating function,
(ieu,) ks
Inn (7 (u)) (o
Σ >0
II
exp (e (12el1))
(330)
ky!
Taking into account the absolute values yields the final result.
This characteristic function bound combined with Theorem 36 gives the following corollary.
Assume as well that [| 2.000.00 (x) S P. for some (N -dependent)
log(s-1) log log(e-T) as N + co. 0o. The Lévy-Prokhorov distance between ,N, , is bounded by:
d
[.log (c-t) 0.99
(pN, =0
+ 16
(331)
Proof. Let T = - From Theorem 36
doog (7)
π (pN.gN) S (
sup
JΦN (u) -N (w)) +
+ +0 (p)..
(332)
T
|
ST
From Lemma 39
sup ST
OoN (w) TN (w)) - x0(c (c
(333)
[all
Similarly,
2d exp (o (log (e ()])
(334)
The second term in Eq. 332 thus dominates, implying the given convergence rate.
N TAIL BOUND LEMMAS
We here prove some helper lemmas giving tail bounds for the marginal entries of 3- Wisihartt ran- dom variables. We will particularly leverage the Bartlett decomposition of (--Wihhrrt matrices (see Appendix A.4). In the following we will use I-) denote vectors the vector space on which the defining representation of HP+1 (IF) acts, with basis (0) J {()))---
We first prove that 3-Wishart matrices are robust to certain low-rank perturbations of their Bartlett decomposition. In particular, we show that removing single column from the Bartlett factor L of a 3-Wishart matrix with many degrees of freedom sreedom-fffectielly effectively reducing the degrees freedom parameter of the matrix by one-has little effeet the joint distribution of entries of the matrix. Intuitively, this allows us to argue in Appendix D.3 that such a Wishart matrix approximately independent from these entries.
58

---

Published as a conference paper at ICLR 2025
Lemma 41 (Robustness of 3-Wishart Bartlett decompositions). Let W be (p 1)+1)= (p 11) 3- Wishart matrix with r degrees of freedom and identity scale matrix. Let LL be the Bartlet decomposition of W. Let L be L with the first column removed, and define W LL For every
t 0,
I
(i W ()) (i) W (b) 2 V28 by ittr S
pexp
vr
t
(JE(L.....))
2
(335)
In particular, the argument converges to under the Ky Fan metric - VF log(pr)
rate
Ω
Proof. Let No.; be the (1,0) entry of L. Note that the No, are i.i.d. The lemma statement is then equivalent to showing that
PP max
No. (VrP+ + 2 +1) S pexp
12
(336)
2
This in turn is implied by:
vr 12
P max lE
|N,,ll 2 BVTE + /23r S pexp
(337)
2
3 i is ddsttttribeeed with 3 degrees of freedom. The union bound along with standard x² tail bounds (see, e.g., Lemma of Laurent & Massart (2000)) thus immediately imply Eq. (337) and therefore also the final result.
We now prove tail bound for a x-distributed random variable with D degrees of freedom. Lemma 42 (Tail bounds for x-distributed random variables). Let x be x-distributed with D degrees of freedom. For every 0 I tt= < D
t +
VD
2 + +++D=+e
s 2exp (-VDi).
(338)
In particular, the argument converges to zero under the Ky Fan metric as - 0o at a rate
VD
Ω
D)
Proof.
We have from standard x² tail bounds (see, e.g., Lemma 1 of Laurent & Massart 2000
that:
P x2 D2 + +2VD exp (-VDt)
(339)
p P DD - -2-2 < eepp (-VD(2)
(340)
This implies that:
x / D+ 2D
P
1 exp (op--vDy (-VDt²
(341)
and (for t < D
IP pxx Dー 2D
< sxxp (-VDR).
D-
(342)
Using the general inequality (for x> -1):
x $1 I
(343)
2'
we have that:
V D 2D + 2D22 VD Dit+
(344)
59

---

Published as a conference paper at ICLR 2025
such that
P x Dit + 1 exp (-VDt)
(345)
Similarly, using the general inequality (for 1 x 1 /t:
2x
1 v1 - Jr,
(346)
we have under the given assumptions that:
4 P DDD - pàt S exp (-VDr))
(347)
Noting that both D + t and D are upper bounded by D + t then implies the final result.
We end with a tail bound for the off-diagonal entries of a B- 3-Wishart matrix.
Lemma 43 (Off- (Off-diagonal 3-Wishart tail bounds). Let W be (p+ ++11 1) (p 1) B- B-Wishart matrix with degrees of freedom and identity scale matrix. There exist universal constants C, K > 0 that depends only on B such that, for every O,
P
[O] W (i] 21 < Kpexp (-C min (t,
(348)
iE!.....p!
Proof. Let LL! be the Bartlett decomposition of W. Let Xo the (0,0) entry of and No,; the (i, 0) entry. The lemma statement is then equivalent to showing that
P
Xo
max VE
|N 2 (1 (1+ < Kpexp (-Cmin (t, (t.
(349)
Note that:
IP
Xo max vE(t.....p))
|o,51 2 (1 (1
< T
max
max
|No.il (1+ (1+1)²
r
i (E(.....p))
(350)
PP
max
X²
max |Nousl
2 21+t
iE (E(...)))))
so by the union bound need only check whether
max PP xa 2 (1 (1+ IP No. 21
- 2
exp (-C min (1, f))
(351)
to prove the final result. As 3 2
2 is ddistrribd with ,3 degrees of freedom the final result immediately follows from standard x2 tail bounds.
60


Reviews:
[{"review_id": "msAzPSsjkT", "reviewer": "Reviewer_TpUa", "summary": "The article defines a quantum neural network (QNN) as a chain of unitary operations applied to an initial quantum state. The loss function is the expectation value of an operator the resulting state with respect to an operator. This definition is common and general. The authors then assume random initialization and show that the loss converges to what they term a Wishart process in the limit of infinitely wide networks. This is in contrast to classical neural nets, where the limit is a Gaussian process (GP).   Having established the Wishart process framework, they use it to formulate several results within it:\n\n(1) While the possibility of QNN convergence to GPs had already been studied in prior cited work, the authors formulate conditions within this framework for the convergence.\n\n(2) Furthermore, they extend and generalize two results on trainability of neural networks from cited articles: \n- Conditions for barren plateaus, i.e. vanishing gradients.\n- The density of local minima is concentrated near the global minimum iff the network is over-parameterized.", "questions": "Can you give more explanation and background on N (equation 25) in the main text?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- The article is very well structured and seems technically of excellent quality. (However, I did not check or verify the proofs in the Appendix in detail) \n- The overall question on the structure of quantum neural networks and their potential advantages is relevant and the new findings advance the field.\nThe new Wishart process framework introduced in this article seems useful, as demonstrated by the results derived mentioned in the summary.", "weaknesses": "- The results are not groundbreaking new but generalizations of previous results and formulated in a more general language and setting.\n- The results are a bit underwhelming, essentially \"it seems unlikely that there exists any computational quantum advantage during the training of QNNs\", which already seemed unlikely before. But this is expected, and this evidence is also valuable, so this is not a strong weakness.\n- The main text is hard to understand without the long Appendix. However, this seems inevitable given the page limit and the subject.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "t3j7omYl7s", "reviewer": "Reviewer_CGjm", "summary": "The authors present a theory study of quantum neural networks showing, in contrast to classical neural network studies, that QNNs obey a so-called Wishart process. In doing so, they derive the existence of barren plateaus and show when a QNN behaves like a Gaussian process.", "questions": "* Where does entanglement fall into these results? One criticism of quantum ML is that entanglement appears not to provide much a benefit in real-world problems. Is there an element of the theory presented here that can touch on this problem?\n\n* What exactly should follow a Wishart distribution in the QNN framework? In a neural network, it is the weights, whereas the NTK will likely look Wishart and the Fisher matrix the same. Purely looking at the math, the NTK and Fisher matrix of a QNN should do the same.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper is written very well and tackles a valid problem. The theoretical analysis appears to be well performed and the authors have done a lot to highlight the connection to prior results.", "weaknesses": "The main weakness I see is in the discussion around limits. It isn't always clear in what limit the authors are discussing the behaviour of the networks. It is also reasonable that a classical neural network's neural tangent kernel matrix has eigenvalues behaving a Wishart distribution and evolving during training. Further, a covariance matrix used to describe a GPR will likely also look very Wishart. While the math and results were presented in the paper, it was never completely clear to me how this all aligned and how I should think about the WIshart process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Buyk7D6fZU", "reviewer": "Reviewer_3K5s", "summary": "It is known that (classical) neural networks typically behave like Gaussian processes in the infinite-width limit under reasonable regularity assumptions. In this work, this result is extended to Quantum Neural Networks which behave like a Wishart process instead. This result is then used to study training, similar to Neural Tangent Kernels in the classical case, and the loss landscape thereof in particular with respect to barren plateus.\n\nWhile the authors should be commended on their opus magnum and an exciting set of ideas / contributions from theoretical physics to machine learning, I share some concerns regarding technical soundness, presentation and topical fit to this publication venue.", "questions": "what is the potential of classical limits within this framework?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 4, "strengths": "1) Several new ideas are introduced from theoretical physics / quantum mechanics and connected with neural networks\n\n2) The claimed result is a major generalization of major theorems in \"classical\" neural networks such as the infinite-width limit of neural networks (Gaussian processes) or Neural Tangent Kernel theory. While several results are derived from this for quantum NNs, there is potential that these ideas can also lead to new results for classical NNs, e.g. via the correspondence principle.", "weaknesses": "1) Technical: The paper lacks basic definitions and introduction of notation, or precision in doing so, that are crucial to follow the train of thought. This is detrimental as several parts of the notation are non-standard, or at least non-standard within the subsections and audiences of this publication venue. For example, in Theorem 1, a notation is introduced as \"....denotes the projection into a (Jordan sub-algebra) A_α ....\", first mention in the introduction around eq 2. Nowhere in the main body is it defined what this precisely means. QM people are left with a vague idea, but not more. Instead, the reader must dig deep into the appendix. It was until around eq 178 in the appendix until I got a better idea, by implication only. Further examples of lack of definition are around eq 5, 7 and 13, or mathematical objects \\mathcal{G} and \\mathcal{H} in Line 291. An example for unclear notation is also in eq 21 in Theorem 2. While non-standard notation is perfectly fine, it doesn't help much if things are kept too informal, even when things \"are easy to see\", and it should defined and introduced properly. This list is non-exhaustive.\n\n2) Presentation: When agreeing to review, I was expecting a 10-page paper in line with the ICLR submission guidelines. While this work technically fulfills these requirements, one can learn and understand very little without working through the 45 page appendix. More honestly, this is a 59-page paper that has been last minute broken into a 10 page teaser with several big claims, many with unclear or vague rationale, followed by the actual paper.\n\nMinor:\nI know this publication venue typically expects numerical experiments, of which there is a lack of any in this paper. From a theoretical physics perspective, this is fine though.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ogX2tKMvlQ", "reviewer": "Reviewer_ZJd3", "summary": "This work establishes a unified theoretical framework for characterizing the trainability behavior of quantum neural networks. It integrates previous findings on barren plateaus and local minima phenomena by introducing a novel class of random processes.", "questions": "1. Is there an algorithm capable of automatically determining the trainability of any given QNNs?\n\n2. Could you explain how to calculate $O^\\alpha$? Are there any efficient algorithms for evaluating these calculations? It would be better if you can provide an example calculation or pseudocode for computing $O^\\alpha$.\n\n3. Previous work in [Anschuetz & Kiani,2022] analyzes local minima phenomena in local and deep, brickwall circuits. Does the current study address these issues? Furthermore, is it applicable to shallow circuits as well?\n\n4. Does this work cover all previous results concerning BP and local minima?  If possible, it would be better to provide a more comprehensive comparison table or discussion section that explicitly shows how their framework subsumes or extends specific prior results on barren plateaus and local minima.\n\n5. Could you clarify the rationale behind using Jordan algebra in this study?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- This paper establishes a unified framework for quantum nerual networks.\n\n- It shows the necessary and sufficient conditions for QNNs to approach the Gaussian process limit, offering comprehensive insights into the trainability of these networks.\n\n- While Jordan algebra is relatively unfamiliar in this domain, the authors have made efforts to clarify its application and relevance to the audience.\n\nOverall, the results presented in this paper are impressive, and I believe they make a valuable contribution to the field of quantum machine learning.", "weaknesses": "- The paper should clearly show its contributions by more explicitly comparing what it offers beyond previous works. For example, it would be beneficial to clarify whether the results offer improvements in handling shallow circuits compared to previous studies in local minima.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)