A Unified Theory of Quantum Neural Network Loss Landscapes

## Abstract

Abstract Classical neural networks with random initialization famously behave as Gaussian processes in the limit of many neurons, which allows one to completely characterize their training and generalization behavior. No such general understanding exists for quantum neural networks (QNNs), which—outside of certain special cases—are known to not behave as Gaussian processes when randomly initialized. We here prove that QNNs and their first two derivatives instead generally form what we call Wishart processes , where certain algebraic properties of the network determine the hyperparameters of the process. This Wishart process description allows us to, for the first time: give necessary and sufficient conditions for a QNN architecture to have a Gaussian process limit; calculate the full gradient distribution, generalizing previously known barren plateau results; and calculate the local minima distribution of algebraically constrained QNNs. Our unified framework suggests a certain simple operational definition for the “trainability” of a given QNN model using a newly introduced, experimentally accessible quantity we call the degrees of freedom of the network architecture.

## I Introduction

### I.1 Motivation

One of the miracles of machine learning on classical computers is that simple, gradient-based optimizers can efficiently find the minimum of extremely high-dimensional, nonconvex loss landscapes, allowing for the efficient training of deep neural networks. Over the past decade this has been understood in more and more detail via random matrix theory. In particular, it is now known that the loss landscapes of randomly initialized, wide neural networks are distributed as Gaussian processes with covariance given by the so-called *neural tangent kernel* (NTK) [, , , ]. The NTK is completely determined by the neural structure of the network, linking the asymptotic behavior of the network to architectural choices made in its construction. This understanding of classical neural networks has been used to show that wide neural networks train efficiently via gradient descent [, , , ]. It also explains other emergent phenomena in deep learning, such as the remarkably good generalization performance of neural networks beyond what learning theory predicts [, ].

One might hope that a similar, universal story would exist for *quantum neural networks* (QNNs). These are classes of neural networks where the associated loss function $\ell$ takes as input a quantum state $\bm{\rho}$, performs some parameterized unitary operation $\bm{U}_{\bm{\theta}}$, and then measures some quantum observable $\bm{O}$. That is,

$$ $\ell\left(\bm{\rho};\bm{\theta}\right)=\Tr\left(\bm{\rho}\bm{U}_{\bm{\theta}}^ {\dagger}\bm{O}\bm{U}_{\bm{\theta}}\right).$ (1) $$

Such networks are defined by the choice of parameterized gate set $\bm{U}_{\bm{\theta}}$ (typically called the *ansatz*) and the observable $\bm{O}$ measured after the ansatz is applied (here called the *objective observable*) [], and are known to efficiently (on a quantum computer) perform learning tasks provably difficult using traditional machine learning methods [, , , , ]. Unfortunately, it is known that QNNs generally do not have a Gaussian process (or *quantum neural tangent kernel (QNTK)*) asymptotic limit [, ]. Indeed, it is not even obvious whether an equivalently simple universal description exists due to QNN training dynamics differing completely in various parameter regimes. For instance, the loss landscapes of generic shallow QNNs are known to be described by so-called *Wishart hypertoroidal random fields* (WHRFs) and dominated by poor local minima [, ]; this is a far cry from the effectively convex training landscapes of deep quantum networks which, in certain circumstances, *can* be shown to have a QNTK limit [, , , , , ]. Unfortunately, these deep networks are still untrainable in practice due to an exponential decay in their gradients known as the *barren plateau* phenomenon [], making estimating gradients on a quantum computer asymptotically intractable(^1^11Girardi and Palma [] also study instances which do not suffer from barren plateaus, but which only achieve an asymptotically-vanishing improvement over random $\bm{\theta}$ in the optimization of Eq. (1).).

The presence of barren plateaus or poor local minima in QNN loss landscapes paints a pessimistic picture for the practical utility of generic QNNs. However, these negative results can be circumvented by considering *structured* QNNs, further complicating any unifying theory of the asymptotic training behavior of QNNs. For instance, $\bm{U}$ may be constrained to belong to some low-dimensional subgroup $G$ of the full unitary group, and $\bm{O}$ in the generating algebra of $G$. This setting is called the *Lie algebra-supported ansatz* (LASA) setting, and is known to be capable of preventing the barren plateau phenomenon from occurring [, ]. Though many of the initial proposals for such structured networks have since been dequantized [, , ], there still do exist unconditionally provable quantum advantages in using such networks for certain learning tasks [, ]. Though this is perhaps the most promising direction in QNN research, nothing concretely is known about the loss landscapes of such networks beyond the variance of the loss function over parameter space.

Figure: Figure 1: Relation between models for quantum variational loss landscapes. Our introduced theory for the loss landscapes of quantum neural networks (QNNs) are *Jordan algebraic Wishart systems* (JAWS), which relate the algebraic structure of a given QNN architecture to an asymptotic random process description of the loss landscape. This JAWS description reduces to the previously-studied Wishart hypertoroidal random fields (WHRFs) and quantum neural tangent kernel (QNTK) in different settings, and also reproduces the loss function variance known for Lie algebra-supported ansatzes (LASAs) and matchgate (MG) networks. Cartoons of the loss landscapes associated with previously studied models are shown.
Refer to caption: x1.png

### I.2 Contributions

Fully understanding how the various phenomenologies of QNN loss landscapes are related is important if we ever hope to have as deep an understanding of QNNs as the neural tangent kernel has enabled for classical neural networks. Motivated by this, we here for the first time prove a concise asymptotic limit for the loss functions of effectively all QNNs with approximately uniformly random initialization, and show it unifies much of our previous understanding of QNN training behavior. We then use this new asymptotic description to prove a variety of novel results on the asymptotic training behavior of QNNs.

We achieve this by demonstrating that all QNNs have a natural algebraic structure described by a *Jordan algebra* $\mathcal{A}$: an abstract algebra equipped with a commutative product, such as the anticommutator $\left\{a,b\right\}=ab+ba$ on $N\times N$ complex Hermitian matrices $a,b\in\mathcal{H}^{N}\left(\mathbb{C}\right)$. In particular, we consider the subalgebra $\mathcal{A}\subseteq\mathcal{H}^{N}\left(\mathbb{C}\right)$ generated by $\left\{\bm{U}_{\bm{\theta}}^{\dagger}\bm{O}\bm{U}_{\bm{\theta}}\right\}_{\bm{
\theta}}$ under the anticommutator. The Jordan subalgebras of $\mathcal{H}^{N}\left(\mathbb{C}\right)$ have been completely classified in the sense that they are all of the form:

$$ $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha},$ (2) $$

where for our purposes each $\mathcal{A}_{\alpha}$ is isomorphic to the algebra of $N_{\alpha}\times N_{\alpha}$ Hermitian matrices $\mathcal{H}^{N_{\alpha}}\left(\mathbb{F}_{\alpha}\right)$ over a field $\mathbb{F}_{\alpha}$ that is one of $\mathbb{R},\mathbb{C},\mathbb{H}$ (labeled by $\beta_{\alpha}=1,2,4$, respectively). This allows us to show that the loss functions of randomly initialized QNNs have a concise asymptotic description in terms of *Wishart-distributed random matrices* over $\mathbb{F}_{\alpha}$:

$$ $\bm{W}_{\alpha}=\bm{X}_{\alpha}\bm{X}_{\alpha}^{\dagger},$ (3) $$

where $\bm{X}_{\alpha}$ is $N_{\alpha}\times r_{\alpha}$ with i.i.d. standard Gaussian entries over $\mathbb{F}_{\alpha}$. $r_{\alpha}$ is called the *degrees of freedom* of $\bm{W}_{\alpha}$, and we find that it dictates the asymptotic behavior of the loss landscape. In short, shifting the landscape so $\bm{O}\succeq\bm{0}$, we prove the asymptotic convergence to a *Wishart process*:

$$ $\ell\left(\bm{\rho}\right)\rightsquigarrow\sum_{\alpha}r_{\alpha}^{-1}\Tr\left (\bm{\rho}^{\alpha}\bm{W}_{\alpha}\right),$ (4) $$

where $\cdot^{\alpha}$ denotes projection into $\mathcal{A}_{\alpha}$,

$$ $r_{\alpha}=\left\lfloor\frac{\Tr\left(\bm{O}^{\alpha}\right)^{2}}{\Tr\left( \left(\bm{O}^{\alpha}\right)^{2}\right)}\right\rceil\sim\rank\left(\bm{O}^{ \alpha}\right),$ (5) $$

and the $\bm{W}_{\alpha}$ are Wishart matrices over $\mathbb{F}_{\alpha}$ with $r_{\alpha}$ degrees of freedom. We also prove similar results demonstrating the asymptotic convergence of the *joint* distribution of the loss with its gradient and Hessian to those of a Wishart process. This allows us to reason about the loss *landscape*, rather than just the loss at a single point or averaged over the landscape. We call this connection between the algebraic structure of a given QNN and its asymptotic Wishart process description a *Jordan algebraic Wishart system* (JAWS). We summarize in Fig. 1 how this JAWS description generalizes previous models of QNN loss landscapes which only held in specific parameter regimes.

The JAWS framework allows us to prove many new, general properties of QNNs. First, we prove a generalized barren plateau result:

$$ $\operatorname{Var}\left[\ell\right]=\sum_{\alpha}\frac{\Tr\left(\left(\bm{O}^{ \alpha}\right)^{2}\right)\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)}{ \dim_{\mathbb{R}}\left(\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right) \right)},$ (6) $$

where $\dim_{\mathbb{R}}\left(\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)$ is the dimension of the automorphism group of $\mathcal{A}_{\alpha}$ as a real manifold. This extends previously known barren plateau results proved in the LASA framework [, ] to the setting where neither $\bm{\rho}$ nor $\bm{O}$ are elements of the algebra generating $\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)$. It also captures the gradient scaling in the setting of so-called matchgate networks [], unifying barren plateau results beyond just the LASA setting.

We are also able to calculate the asymptotic density of local minima of a QNN:

$$ $\kappa\left(E\right)=\operatorname*{\scalerel*{\ast}{\sum}}_{\alpha}f_{\Gamma} \left(E;k_{\alpha},\theta_{\alpha}\right);$ (7) $$

i.e., we show that it is a convolution of gamma distributions with shape and scale parameters $k_{\alpha}$ and $\theta_{\alpha}$, respectively, which we calculate. The $k_{\alpha}$ and $\theta_{\alpha}$ are such that $\kappa\left(E\right)$ experiences a phase transition: local minima of a network with $p$ parameters are concentrated near the global minimum if and only if the network is *overparameterized*, which occurs when:

$$ $p\geq\max_{\alpha}\beta_{\alpha}r_{\alpha}.$ (8) $$

This result generalizes previous studies of local minima in QNNs [, ] to a setting where the variational ansatz may have some sort of algebraic structure.

Finally, we prove the necessary and sufficient conditions for a class of QNNs to asymptotically converge to a neural tangent kernel limit; we then show that when such a limit exists, the covariance over a data set $\mathcal{R}$ of inputs $\bm{\rho}$ is an explicit quadratic expression of the $\bm{\rho}$. This expression simplifies dramatically when the $\bm{\rho}\in\mathcal{R}$ mutually commute:

$$ $\operatorname{Cov}\left(\bm{\rho},\bm{\rho^{\prime}}\right)=\sum\limits_{ \alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R }}\left(\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)}\Tr\left( \bm{\rho}^{\alpha}\bm{\rho^{\prime}}^{\alpha}\right).$ (9) $$

This form of the covariance can then be immediately fed into classical neural tangent kernel results to reason about the training behavior [, , , ] and generalization ability [, ] of such networks.

Taken together, our results indicate that QNNs are efficiently trainable asymptotically using problem-independent optimization algorithms if and only if they satisfy two properties:

- 1.
*Absence of barren plateaus*:
$\sum_{\alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(
\left(\bm{\rho}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(
\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)}=\operatorname{
\Omega}\left(\operatorname{poly}\left(\log\left(N\right)\right)^{-1}\right).$
(10)
- 2.
*Absence of poor local minima*:
$p\geq\max_{\alpha}\beta_{\alpha}r_{\alpha}.$
(11)

### I.3 Outline

To make this work easier to follow we give an informal discussion of our results and their proofs in the main text, and leave a formal discussion—which can be read independently—for the appendix.

In Sec. II we give some background and fix notation for the main text. In Sec. III we show that quantum neural networks can generally be described completely in terms of Jordan algebras. In Sec. IV we present our main result, the asymptotic description of the joint distribution of the loss with its gradient and Hessian in terms of Wishart-distributed random variables. We discuss new results in loss landscape theory that can be deduced from our main results—in particular, generalizations of barren plateau, local minima, and quantum neural tangent kernel theory—in Sec. V. We then conclude with a high-level discussion on the implications for quantum machine learning in Sec. VI.

The main results discussed in Secs. III and IV are formally stated—with an explicit listing of formal assumptions—in Appendix B. Definitions and background necessary for these formal theorem and assumption statements are laid out in Appendix A. Formal proofs of the main results are given in Appendix D. Formal statements and proofs of the corollaries discussed in Sec. V are given in Appendix C. A discussion of open directions in the research of quantum neural network loss landscapes—particularly, a discussion of settings where our results do not hold—is given in Appendix E.

The remainder of the Appendices—those after the references—prove claims and helper lemmas used in Appendices B, C, and D.

## II Preliminaries

### II.1 Quantum Neural Networks

We first review *quantum neural networks* (QNNs). These are defined by a parameterized *ansatz*

$$ $\bm{U}_{\bm{\theta}}=\bm{g}_{0}\prod_{i=p}^{1}\bm{g}_{i}\exp\left(-\mathrm{i} \theta_{i}\bm{A}_{i}\right)\bm{g}_{i}^{\dagger}\bm{h},$ (12) $$

where $\bm{A}_{i}$ is a complex Hermitian $N\times N$ matrix and the $\bm{g}_{i},\bm{h}$ $N\times N$ unitary matrices. Generally, the $\bm{g}_{i},\bm{h},\exp\left(-\mathrm{i}\theta_{i}\bm{A}_{i}\right)$ may be constrained to belong to a path-connected subgroup $G$ of the unitary group $\operatorname{U}\left(N\right)$ to strengthen the inductive bias of the network []. Such a constraint has also been used as a theoretical model for shallow quantum networks [].

Training such networks involves minimizing an empirical risk of the form:

$$ $f\left(\bm{\theta}\right)=\frac{1}{\left\lvert\mathcal{R}\right\rvert}\sum_{ \bm{\rho}\in\mathcal{R}}\ell\left(\bm{\rho};\bm{\theta}\right)=\frac{1}{\left \lvert\mathcal{R}\right\rvert}\sum_{\bm{\rho}\in\mathcal{R}}\Tr\left(\bm{U}_{ \bm{\theta}}\bm{\rho}\bm{U}_{\bm{\theta}}^{\dagger}\bm{O}\right).$ (13) $$

Here, $\mathcal{R}$ can be thought of a data set comprising multiple input *quantum states* $\bm{\rho}$—that is, $N\times N$ positive semidefinite, complex-valued Hermitian matrices with trace $1$—and $\bm{O}$ a complex Hermitian $N\times N$ matrix. When $\bm{O}$ and the algebra elements generating $\bm{U}_{\bm{\theta}}$ can be expressed as a sum of $\operatorname{O}\left(\operatorname{poly}\log\left(N\right)\right)$ Pauli matrices, such a loss can be estimated in $\operatorname{O}\left(\operatorname{poly}\log\left(N\right)\right)$ time on a quantum computer [].

Our first main result is that all losses of the form of $\ell\left(\bm{\rho};\bm{\theta}\right)$ can be interpreted in terms of automorphisms of Jordan algebras. In preparation for discussing this connection we now review Jordan algebras.

### II.2 Jordan Algebras

A *Jordan algebra* over the reals is formally a real vector space $V$ with a commutative multiplication operation $\circ$ acting on $u,v\in V$ satisfying the *Jordan identity*:

$$ $u\circ\left(\left(u\circ u\right)\circ v\right)=\left(u\circ u\right)\circ \left(u\circ v\right),$ (14) $$

which ensures the associativity of the power. A simple example is the Jordan algebra $\mathcal{H}^{N}\left(\mathbb{C}\right)$ of $N\times N$ complex-valued Hermitian matrices with $\circ$ given by the anticommutator. The Jordan algebra of Hermitian matrices is *Euclidean* as it satisfies the defining property []:

$$ $\forall A,B\in\mathcal{A},\,A\circ A+B\circ B=0\iff A=B=0.$ (15) $$

It is apparent that all Jordan subalgebras of a Euclidean Jordan algebra are also Euclidean.

The Euclidean Jordan algebras $\mathcal{A}$ have been completely classified [] and are known to have a semisimple decomposition:

$$ $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha},$ (16) $$

where $\mathcal{A}_{\alpha}$ is isomorphic to one of the simple Jordan algebras. For our purposes, each $\mathcal{A}_{\alpha}$ is isomorphic to one of three: $N_{\alpha}\times N_{\alpha}$ Hermitian matrices over a field $\mathbb{F}_{\alpha}$ that is one of the reals, complex numbers, or quaternions. We label these three cases with the integers $\beta_{\alpha}=1,2,4$, respectively. The automorphism group $\operatorname{Aut}\left(\mathcal{A}\right)$ of such an algebra is also known; its path-connected components are isomorphic to direct products of the classical Lie groups $\operatorname{SO}\left(N_{\alpha}\right)$, $\operatorname{SU}\left(N_{\alpha}\right)$, and $\operatorname{Sp}\left(N_{\alpha}\right)$ [, ]. Finally, it is known that the Frobenius inner product $\left\langle a,b\right\rangle_{R_{\alpha}}$ between $a,b\in\mathcal{A}_{\alpha}$ in any representation $R_{\alpha}$ is always proportional to the Frobenius inner product in the defining representation $D_{\alpha}$ []:

$$ $\left\langle a,b\right\rangle_{R_{\alpha}}=I_{\alpha}\left\langle a,b\right \rangle_{D_{\alpha}},$ (17) $$

where $I_{\alpha}>0$ depends only on $R_{\alpha}$. We refer the interested reader to Appendix A.2 for more details on Jordan algebras, as well as details on some subtleties in these classification results.

## III Quantum Neural Networks Have Jordan Algebraic Descriptions

Recall the general form of a QNN loss function:

$$ $\ell\left(\bm{\rho};\bm{\theta}\right)=\Tr\left(\bm{\rho}\bm{U}_{\bm{\theta}}^ {\dagger}\bm{O}\bm{U}_{\bm{\theta}}\right),$ (18) $$

where

$$ $\bm{U}_{\bm{\theta}}=\bm{g}_{0}\prod_{i=p}^{1}\bm{g}_{i}\exp\left(-\mathrm{i} \theta_{i}\bm{A}_{i}\right)\bm{g}_{i}^{\dagger}\bm{h}$ (19) $$

generally may be such that $\bm{g}_{i},\bm{h},\exp\left(-\mathrm{i}\theta_{i}\bm{A}_{i}\right)$ belong to some path-connected subgroup $G\subseteq\operatorname{U}\left(N\right)$.

The $\bm{U}_{\bm{\theta}}^{\dagger}\bm{O}\bm{U}_{\bm{\theta}}$ generate some Jordan subalgebra $\mathcal{A}\subseteq\mathcal{H}^{N}\left(\mathbb{C}\right)$ with Jordan product induced by $\mathcal{H}^{N}\left(\mathbb{C}\right)$. Due to this fact, as well as the classification results discussed in Sec. II.2, we may equivalently consider Eq. (18) in the following way:

- 1.
$\bm{O}\in\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha}$.
- 2.
$\bm{U}_{\bm{\theta}}\in G=\bigtimes_{\alpha}G_{\alpha}$, where $G$ is isomorphic to a subgroup of $\operatorname{Aut}\left(\mathcal{A}\right)$ and each $G_{\alpha}$ is isomorphic to one of $\operatorname{SO}\left(N_{\alpha}\right),\operatorname{SU}\left(N_{\alpha}
\right),\operatorname{Sp}\left(N_{\alpha}\right)$.
- 3.
Denoting by $\cdot^{\alpha}$ the projection of $\cdot$ into the defining representation of $\mathcal{A}_{\alpha}$ with associated trace $\Tr_{\alpha}\left(\cdot\right)$,
$\ell\left(\bm{\rho};\bm{\theta}\right)=\sum_{\alpha}I_{\alpha}\Tr_{\alpha}
\left(\bm{\rho}^{\alpha}\bm{U}_{\bm{\theta}}^{\alpha\dagger}\bm{O}^{\alpha}\bm
{U}_{\bm{\theta}}^{\alpha}\right).$
(20)

We discuss the universality of this Jordan algebraic description in more detail in Appendix B.1.

This description of QNN loss landscapes in terms of Jordan algebraic properties allows us to give a simple way to classify various QNN architectures; we call this collection of algebraic objects a *Jordan algebraic Wishart system* (JAWS). We will next tie this classification to the asymptotic properties of the loss landscape through the use of Wishart random matrices, justifying the use of “Wishart” in the nomenclature.

## IV Quantum Neural Networks Are Wishart Processes

Our main result is proving that wide QNNs form *Wishart processes*. Such processes can be written exactly in terms of the matrix elements of Wishart-distributed positive semidefinite random matrices $\bm{W}$, which are distributed as:

$$ $\bm{W}=\bm{X}\bm{X}^{\dagger}$ (21) $$

for $\bm{X}$ a rectangular matrix with i.i.d. standard normal entries over a field $\mathbb{F}$ (assumed $\mathbb{R}$ if not otherwise stated). The number of columns $r$ of $\bm{X}$ is called the *degrees of freedom* of $\bm{W}$ in analogy with the degrees of freedom of a $\chi^{2}$-distributed random variable; indeed, the diagonal entries of $\bm{W}$ are (up to a constant scaling) i.i.d. $\chi^{2}$-distributed with $\beta r$ degrees of freedom.

Our main result is as follows, stated informally here with a formal statement deferred to Appendix B.2:

###### Theorem 1 (Quantum neural networks are Wishart processes, informal) .

Consider a QNN with associated algebraic description as in Sec. III, initialized approximately uniformly at random. Let $\ell^{\ast}$ be the optimum of the loss and $\overline{o}_{\alpha}$ the mean eigenvalue of $\bm{O}^{\alpha}$, where $\cdot^{\alpha}$ denotes the projection into $\mathcal{A}_{\alpha}$. Then, there is a convergence in joint distribution over $\bm{\rho}$ at any $\bm{\theta}$:

$$ $\ell\left(\bm{\rho};\bm{\theta}\right)-\ell^{\ast}\rightsquigarrow\sum_{\alpha }\hat{\ell}_{\alpha}\left(\bm{\rho}^{\alpha}\right)\equiv\sum_{\alpha}\frac{I_ {\alpha}\overline{o}_{\alpha}}{r_{\alpha}}\Tr_{\alpha}\left(\bm{\rho}^{\alpha} \bm{W}_{\alpha}\right)$ (22) $$

which happens polynomially quickly in the dimensions of the $\mathcal{A}_{\alpha}$. The $\bm{W}_{\alpha}$ are each independent Wishart-distributed random matrices in the defining representation of $\mathcal{A}_{\alpha}$ with

$$ $r_{\alpha}=\left\lfloor\frac{\Tr_{\alpha}\left(\bm{O}^{\alpha}\right)^{2}}{\Tr _{\alpha}\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}\right\rceil$ (23) $$

degrees of freedom.

An illustration of this distribution is provided in Fig. 2(a). The proof follows by using the fact that the marginal distributions of matrix elements of the ansatz are approximately Gaussian distributed; this is made quantitative by bounding the errors in the associated joint characteristic functions over the inputs $\bm{\rho}$. This is enough to demonstrate convergence of each $\hat{\ell}_{\alpha}\left(\bm{\rho}\right)$ to a convolution of Wishart processes. We then prove a Welch–Satterthwaite-like result to demonstrate an asymptotic convergence of this convolution to a single Wishart process with a degrees of freedom parameter given by Eq. (23). The full proof is given in Appendix D. We there also prove a strengthened result—convergence *pointwise* in the corresponding probability densities—under certain additional assumptions on the initialization of the network.

Figure: Figure 2: Loss and derivative densities. (a) A cartoon of the loss density when the quantum neural network has a *pure input* (i.e., rank-$1$) and when it has a *mixed input* (i.e., of rank greater than $1$). The distributions are centered at the mean eigenvalue of the objective observable. The mixed input density also illustrates the density when there are many algebraic sectors and the input is mixed when projected into any one of them. (b) A cartoon of the gradient density conditioned on a nonzero loss function value. The distribution is centered at zero.
Refer to caption: x2.png

We not only prove the convergence of the QNN loss function $\ell$ to that of a Wishart process, but also $\ell$, its gradient, and Hessian jointly. In doing so we are able to show that the *loss landscape* of QNNs—not just the landscape at a single point in parameter space, or averaged over it—converges to that of a Wishart process. We here only give an informal statement of the result in the special case when each $\bm{\rho}^{\alpha}$ is rank-$1$, but give the full distribution and formal statement in Appendix B.2 with proof in Appendix D.

###### Theorem 2 (Gradient distribution, informal) .

Consider the setting of Theorem 1. Assume each $\bm{\rho}^{\alpha}$ is rank-$1$ in its defining representation. Let $\sigma_{o}^{\alpha}$ be the standard deviation of the eigenvalues of $\bm{O}^{\alpha}$. The joint distribution of $\partial_{\theta_{i}}\ell\left(\bm{\rho};\bm{\theta}\right)$ conditioned on $\ell_{\alpha}\left(\bm{\rho};\bm{\theta}\right)=z_{\alpha}$ asymptotically converges to the joint distribution of

$$ $\hat{\ell}_{;i\mid\left\{z_{\alpha}\right\}}=\sum_{\alpha}\hat{\ell}_{;i\mid \left\{z_{\alpha}\right\}}^{\alpha}\equiv\sum_{\alpha}\frac{2I_{\alpha}\sigma_ {o}^{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right)}{N_{\alpha}}\sqrt{ \frac{\beta_{\alpha}z_{\alpha}}{I_{\alpha}\overline{o}_{\alpha}}}G_{i}^{\alpha }\chi_{i}^{\alpha}$ (24) $$

polynomially quickly in the dimensions of the $\mathcal{A}_{\alpha}$. Here, the $\chi_{i}^{\alpha}$ are independent $\chi$-distributed random variables with $\max\left(2,\beta_{\alpha}\right)$ degrees of freedom and the $G_{i}^{\alpha}$ are i.i.d. standard normal random variables.

This distribution at finite $z_{\alpha}$ is illustrated in Fig. 2(b).

We also prove the Hessian distribution at critical points. We informally report the result when each $\bm{\rho}^{\alpha}$ is rank-$1$ and the spectrum of each $\bm{O}^{\alpha}$ is sufficiently concentrated around its mean; this latter condition is typical of low-weight fermionic [] and local spin Hamiltonians []. The full distribution and formal statement is once again given in Appendix B.2 with proof in Appendix D.

###### Theorem 3 (Hessian distribution, informal) .

Consider the setting of Theorem 2. Assume as well $\frac{\sigma_{o}^{\alpha}}{\overline{o}_{\alpha}}\to 0$ for all $\alpha$. The joint distribution of $\partial_{\theta_{i}}\partial_{\theta_{j}}\ell\left(\bm{\rho};\bm{\theta}\right)$ conditioned on $\ell_{\alpha}\left(\bm{\rho};\bm{\theta}\right)=z_{\alpha}$ and $\hat{\ell}_{;i\mid\left\{z_{\alpha}\right\}}^{\alpha}=0$ asymptotically converges to the joint distribution of

$$ $\hat{\ell}_{;i,j\mid\left\{z_{\alpha}\right\},0}=\sum_{\alpha}\frac{2I_{\alpha }\sigma_{o}^{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right)}{N_{\alpha}^{2 }}\sqrt{\frac{z_{\alpha}}{I_{\alpha}\overline{o}_{\alpha}}}G_{i}^{\alpha}\chi_ {j}^{\alpha}W_{i,j}^{\alpha}$ (25) $$

polynomially quickly in the dimensions of the $\mathcal{A}_{\alpha}$. Here, the $\chi_{i}^{\alpha}$ are independent $\chi$-distributed random variables with $\max\left(2,\beta_{\alpha}\right)$ degrees of freedom and the $G_{i}^{\alpha}$ are i.i.d. standard normal random variables. The $\bm{W}^{\alpha}$ are independent Wishart-distributed random matrices with $\beta_{\alpha}r_{\alpha}$ degrees of freedom.

Both of these derivative results follow by carefully considering the joint distribution of elements of a Wishart-distributed random matrix via the use of the *Bartlett decomposition* of Wishart-distributed matrices. Further simplifications follow by bounding the error in probability between various products of these marginal elements to simpler distributions.

###### Theorem 1 (Quantum neural networks are Wishart processes, informal) .

###### Theorem 2 (Gradient distribution, informal) .

###### Theorem 3 (Hessian distribution, informal) .

## V New Results in Landscape Theory From the JAWS Framework

With the full asymptotic distribution of QNN loss landscapes in hand, we are now able to prove several novel results in QNN loss landscape theory.

### V.1 Barren Plateaus

The first implication of our results that we will discuss is the unification of barren plateau results [, ] in the large $N$ limit.

###### Corollary 4 (General expression for the loss function variance, informal) .

Let $\ell$ be as in Theorem 1. The variance of the loss over the initialization is:

$$ $\operatorname{Var}\left[\ell\left(\bm{\rho}\right)\right]=\sum_{\alpha}\frac{ \Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(\left(\bm{\rho}^{ \alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(\operatorname{Aut}\left( \mathcal{A}_{\alpha}\right)\right)}.$ (26) $$

This follows immediately from the variance of elements of Wishart-distributed random matrices; see Appendix C.1 for more details. Despite the simplicity of the proof (given Theorem 1), this result generalizes Theorem 1 of Ragone *et al.* [], which only considered when either $\mathrm{i}\bm{O}$ or $\mathrm{i}\bm{\rho}$ was in the algebra generating $\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)$. However, our result holds for *all* variational ansatzes due to our general Jordan algebraic formulation of QNNs and our use of projections into simple Jordan algebras rather than simple Lie algebras.

###### Corollary 4 (General expression for the loss function variance, informal) .

### V.2 The Quantum Neural Tangent Kernel

We now connect our results to the quantum neural tangent kernel (QNTK) literature [, , , , , ]. The landmark result in this field is that, in certain settings, QNN loss functions are asymptotically Gaussian processes. However, this same body of work has noted that such a Gaussian process description cannot generally be true. For instance, if the objective observable $\bm{O}$ is rank-$1$ and the algebra $\mathcal{A}$ associated with the QNN is the space of complex Hermitian matrices, it is known that the loss is proportional to an exponentially-distributed random variable [].

Our results can be seen as a unifying model of neural network loss landscapes, including both when convergence to a Gaussian process is achieved and when it is not. Recall that Theorem 1 demonstrated the asymptotic expression for the QNN loss:

$$ $\hat{\ell}\left(\bm{\rho}\right)=\sum_{\alpha}\frac{I_{\alpha}\overline{o}_{ \alpha}}{r_{\alpha}}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\bm{W}^{\alpha}\right).$ (27) $$

This correctly captures the exponential behavior when $\mathcal{A}$ is the space of complex Hermitian matrices and $\rank\left(\bm{O}\right)=r=1$. This is because the diagonal entries of such a complex Wishart matrix are (up to a constant) $\chi^{2}$-distributed with two degrees of freedom each, which is identical to an exponential distribution.

Indeed, our more general result can be used to *exactly* characterize when QNNs asymptotically form Gaussian processes. This occurs when Eq. (27) has variance asymptotically vanishing less quickly than its higher-order cumulants; equivalently, if there exists some $\mathcal{N}\geq 1$ such that $\mathcal{N}\hat{\ell}\left(\bm{\rho}\right)$ has asymptotically nonvanishing variance yet vanishing higher-order cumulants. As the cumulants of Wishart matrix elements are known, one can show (see Appendix C.2) that higher-order cumulants vanish for any:

$$ $\mathcal{N}\leq\operatorname{o}\left(\min\limits_{\alpha}\frac{r_{\alpha}^{ \frac{2}{3}}}{I_{\alpha}\overline{o}_{\alpha}\Tr_{\alpha}\left(\left(\bm{\rho} ^{\alpha}\right)^{3}\right)^{\frac{1}{3}}}\right).$ (28) $$

We thus can state this pair of conditions as follows.

###### Corollary 5 (Exact conditions for convergence to a Gaussian process, informal) .

Let $\ell$ be as in Theorem 1. $\mathcal{N}\ell\left(\bm{\rho}\right)$ is asymptotically a Gaussian process if and only if Eq. (28) is satisfied and $\mathcal{N}^{2}\operatorname{Var}\left[\ell\left(\bm{\rho}\right)\right]$ (as in Corollary 4) is nonvanishing for all $\bm{\rho}\in\mathcal{R}$.

When these conditions are met, the covariance is given by:

$$ $\displaystyle\mathcal{K}\left(\bm{\rho},\bm{\rho^{\prime}}\right)$ $\displaystyle=\lim_{N\to\infty}\mathcal{K}_{N}\left(\bm{\rho},\bm{\rho^{\prime }}\right)$ (29) $\displaystyle=\lim_{N\to\infty}\mathcal{N}^{2}\sum_{\alpha}\frac{\beta_{\alpha }\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}{2\dim_{\mathbb{R}}\left( \operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)r_{\alpha}}I_{\alpha }\mathbb{E}\left[\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\left(\bm{W}^{\alpha}-r_{ \alpha}\right)\right)\Tr_{\alpha}\left(\bm{\rho^{\prime}}^{\alpha}\left(\bm{W} ^{\alpha}-r_{\alpha}\right)\right)\right].$ $$

It is now easy to see why (for simple $\mathcal{A}$) the $r=1$ case does not converge to a Gaussian process: in this setting we may only choose a normalization $\mathcal{N}\leq\operatorname{o}\left(N\right)$ such that Eq. (28) is satisfied and higher-order cumulants vanish, but here the variance is vanishing.

In Appendix C.2 we are also more explicit on the form of Eq. (29). We use known results for the mixed cumulants of Wishart matrix elements to show that the covariance can be written as an explicit, quadratic expression in $\bm{\rho}^{\alpha}$ and $\bm{\rho^{\prime}}^{\alpha}$, depending only on $\mathcal{A}$. We further show in Appendix C.2 that, when the $\bm{\rho}\in\mathcal{R}$ can be mutually diagonalized,

$$ $\mathcal{K}_{N}\left(\bm{\rho},\bm{\rho^{\prime}}\right)=\mathcal{N}^{2}\sum \limits_{\alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}{\dim_{ \mathbb{R}}\left(\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)} \Tr\left(\bm{\rho}^{\alpha}\bm{\rho^{\prime}}^{\alpha}\right).$ (30) $$

This form of the covariance can then be immediately fed into neural tangent kernel results to reason about the training behavior [, , , ] and generalization ability [, ] of such networks. For instance, by the results of Girardi and Palma [], this covariance suggests that gradient descent does not reach the global minimum in time polynomial in the model size for unstructured QNNs. This is discussed in more detail in Appendix C.2.

###### Corollary 5 (Exact conditions for convergence to a Gaussian process, informal) .

### V.3 Local Minima

We end by examining the distribution of local minima of the loss landscape. This has been done previously in the more restricted setting where no structure is imposed on the ansatz or loss function [, ]. In this section we assume the setting of Theorem 3, as well as assume that all

$$ $\gamma_{\alpha}\equiv\frac{p}{\beta_{\alpha}r_{\alpha}}$ (31) $$

are held constant as $N\to\infty$. The $\gamma_{\alpha}$ are generalizations of the *overparameterization ratio* discussed in Anschuetz [] and Anschuetz and Kiani [], where now the QNN may have some algebraic structure.

We calculate the distribution of local minima using the *Kac–Rice formula* [], which gives the expected density of local minima of a random field $\ell$ at a function value $z$. Anschuetz [] demonstrated that the assumptions for the Kac–Rice formula are satisfied for variational loss landscapes, and when rotationally invariant on the $p$-torus takes the form:

$$ $\mathbb{E}\left[\operatorname{Crt}_{0}\left(z\right)\right]=\left(2\uppi\right )^{p}\mathbb{E}\left[\det\left(\bm{H}_{z}\right)\bm{1}\left\{\bm{H}_{z}\succeq \bm{0}\right\}\right]\mathbb{P}\left[\bm{G}_{z}=\bm{0}\right]\mathbb{P}\left[ \ell=z\right].$ (32) $$

Here, $\mathbb{E}\left[\operatorname{Crt}_{0}\left(z\right)\right]$ is the expected density of local minima at a function value $z$, $\bm{G}_{z}$ is the gradient conditioned on $\ell=z$, and $\bm{H}_{z}$ is the Hessian conditioned on $\ell=z$ and $\bm{G}=\bm{0}$. In a slight abuse of notation, here $\mathbb{P}\left[\cdot\right]$ denotes the probability density associated with the event $\cdot$.

We can evaluate this expression using Theorems 1, 2, and 3. To simplify the expression here, we assume the addition of a regularization term of the form:

$$ $R_{L}\left(\bm{\theta}\right)=L\left\lVert\bm{\theta}-\bm{C}\right\rVert_{2}^{2}$ (33) $$

to the loss; we also present only the relative density rather than the total number of minima at a loss function value $z$. The full expression counting local minima—both with and without the regularization of Eq. (33)—are described in detail in Appendix C.3.

###### Corollary 6 (Density of local minima, informal) .

Consider the setting of Theorem 3 with an additional regularization term of the form of Eq. (33). The density of local minima at a loss function value $z>0$ is, to multiplicative leading order,

$$ $\kappa\left(z\right)=\operatorname*{\scalerel*{\ast}{\sum}}_{\alpha:\gamma_{ \alpha}<1}f_{\Gamma}\left(\frac{z}{\overline{o}_{\alpha}\Tr\left(\bm{\rho}^{ \alpha}\right)};\frac{\beta_{\alpha}r_{\alpha}}{2},\frac{2}{\beta_{\alpha}r_{ \alpha}}\right).$ (34) $$

Here, $f_{\Gamma}\left(\cdot;k,\theta\right)$ denotes the gamma distribution with shape $k$ and scale $\theta$ parameters:

$$ $f_{\Gamma}\left(x;k,\theta\right)=\frac{1}{\operatorname{\Gamma}\left(k\right) \theta^{k}}x^{k-1}\exp\left(-\frac{x}{\theta}\right).$ (35) $$

Figure: Figure 3: Density of local minima in various parameter regimes. The density of local minima when the quantum neural network is *underparameterized*, *overparameterized*, and when some algebraic sectors are underparameterized and some are overparameterized (*mix of sectors*). In the latter case, convolutions between various algebraic sectors broaden the distribution, though the proportion of minima near the global minimum in energy still vanishes exponentially in the system size due to the exponential tails of the gamma distribution.
Refer to caption: x3.png

Examples of this distribution are plotted in Fig. 3, where we give a finite width to the density in the *overparameterized regime*—where all $\gamma_{\alpha}\geq 1$—to represent potential finite-size effects. Intriguingly, the variance of this distribution when all $\gamma_{\alpha}<1$—the *underparameterized regime*—corresponds to the variance of the loss function itself, i.e., as given in Corollary 4. However, due to the exponential tails of the gamma distribution, even when there are no barren plateaus in the loss landscape there is only an exponentially small fraction of local minima in the vicinity of the global minimum in this regime. That is, both barren plateaus and poor local minima are potential obstructions to efficient trainability that must be taken into account in the design of practical QNNs.

###### Corollary 6 (Density of local minima, informal) .

## VI Conclusion

Taken together, our results show that the natural model for quantum neural networks is not one of Gaussian processes obeying a quantum neural tangent kernel, but rather one of *Wishart processes*. This Wishart process model unifies all of the recent major thrusts in calculating properties of quantum neural network loss landscapes. Indeed, our results allow us to propose a simple operational definition for the “trainability” of quantum neural networks, which to date has been a term used heuristically without any formal definition(^2^22The concept of “efficient learning” has a previously-introduced formal definition [], which is a nonconstructive statement on whether there exists an efficient algorithm that optimizes a given loss function. Ours is an *operational* definition, in the sense that certain structural properties of a given class of quantum neural networks suggest the class is or is not efficient to train using problem-independent learning algorithms such as gradient descent.).

###### Definition 7 (Trainability of quantum neural networks) .

Consider a QNN in a Hilbert space of dimension $N$ with $p$ trained parameters. Let $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha}$ be the corresponding Jordan algebra as in Sec. III. Define the *degrees of freedom* parameter:

$$ $r_{\alpha}\equiv\frac{\Tr\left(\bm{O}^{\alpha}\right)^{2}}{\Tr\left(\left(\bm{ O}^{\alpha}\right)^{2}\right)}.$ (36) $$

We say that the QNN is *trainable* if and only if, as $N\to\infty$, the QNN satisfies:

- 1.
*Absence of barren plateaus* (Corollary 4):
$\sum_{\alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(
\left(\bm{\rho}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(
\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)\right)}=\operatorname{
\Omega}\left(\operatorname{poly}\left(\log\left(N\right)\right)^{-1}\right).$
(37)
- 2.
*Absence of poor local minima* (Corollary 6):
$p\geq\max_{\alpha}\beta_{\alpha}r_{\alpha}.$
(38)

Practically, where does this leave quantum neural networks? For one, it seems unlikely that there exists any computational quantum advantage during the training of QNNs, outside of HHL-like speedups [, ] and training algorithms that leverage some existing knowledge of the data to be learned [, , ]. This is due to efficient classical simulation algorithms for quantum systems algebraically constrained to explore a low-dimensional subspace [, ], as is required in Definition 7. This was noted in Cerezo *et al.* [] for deep QNNs, where the authors postulated that deep QNNs exhibiting no barren plateaus are classically simulable (outside of certain special cases). This is due to the fact that barren plateaus exist in deep QNNs when the explored Hilbert space grows superpolynomially quickly with the system size, which is typically a prerequisite for demonstrating a superpolynomial quantum computational advantage. Our results demonstrate a similar phenomenon for shallow QNNs: poor local minima in polynomially-sized circuits can only be avoided when the effective Hilbert space dimension grows at most polynomially quickly with the system size. This leaves the space for a practical, superpolynomial quantum advantage when training with a problem-agnostic algorithm such as gradient descent even narrower than previously believed: a veritable Amity Island in a sea of negative results. One ray of hope is the known existence of *polynomial* quantum advantages during inference in such a setting [, ]. Intelligent warm-starting of the optimization procedure may be another way to circumvent poor training [], though more must be done to fully understand how precise warm-starting must be for training to be efficient. Given the exact asymptotic form of the loss landscape we give here, such an analysis may be possible in the future.

Our work gives a unified understanding of quantum neural networks as Wishart processes. Great strides have been made in the classical machine learning literature in understanding the training dynamics [, , , ] and generalization behavior [, ] of classical neural networks via their connections to Gaussian processes, which unfortunately only port over in the specific settings where the Wishart process itself approaches a Gaussian process. Our results encourage an understanding of how specific properties of Wishart processes, not just Gaussian processes, influence the learning behavior of quantum networks in order to more fully grasp how quantum neural networks learn.

###### Acknowledgements.

###### Definition 7 (Trainability of quantum neural networks) .

###### Acknowledgements.

## Appendix A Preliminaries for Formal Discussion of Results

We begin by reviewing concepts that we will use in proving our results. We also give a summary of the notation we use throughout in Table 1.

**Table 1: Table of notation. Notation used in the presentation of our results are given in the left column with corresponding meaning given in the right column. We also note here that exponents written as decimals throughout our results are chosen somewhat arbitrarily, i.e., can be improved by any sufficiently small constant $\epsilon>0$ in the exponent. Finally, we note that we will often forgo writing “…a sequence of [ansatzes, loss functions, observables, QNNs, …]…” when describing our asymptotic convergence results for brevity.**
| $\mathbb{N}_{0}$ | Natural numbers including 0 0 |
| --- | --- |
| $\mathbb{N}_{1}$ | Natural numbers excluding 0 0 |
| $\left[n\right]$ | Natural numbers from $1$ through $n$ |
| $\mathcal{H}^{n}\left(\mathbb{F}\right)$ | Jordan algebra of $n\times n$ Hermitian matrices over $\mathbb{F}$ |
| $\mathbb{F}$ | Field, here one of $\mathbb{R},\mathbb{C},\mathbb{H}$ |
| $\beta$ | $\beta=1,2,4$ when associated field $\mathbb{F}=\mathbb{R},\mathbb{C},\mathbb{H}$, respectively |
| $\dim_{\mathbb{F}}\left(\cdot\right)$ | Dimension of $\cdot$ as a vector space over $\mathbb{F}$ |
| $\mathcal{N}^{\beta}\left(0,1\right)$ | Standard normal distribution over $\mathbb{F}$ (given by $\beta$) |
| $\mathcal{W}_{n}^{\beta}\left(r,\bm{\varSigma}\right)$ | $\beta$-Wishart distribution of $n\times n$ matrices with $r$ degrees of freedom<br>and scale matrix $\bm{\varSigma}$ |
| $\rightsquigarrow$ | Convergence in distribution |
| $\xrightarrow{p}$ | Convergence in probability |
| $\odot$ | Hadamard product |
| $\mathcal{A}_{\alpha}$ | Simple components of semisimple Jordan algebra $\mathcal{A}$ |
| $G_{\alpha}$ | Lie group isomorphic to a connected component of $\operatorname{Aut}\left(\mathcal{A}_{\alpha}\right)$ |
| $\mathfrak{g}_{\alpha}$ | Lie algebra generating $G_{\alpha}$ |
| $\cdot^{\alpha}$ | Defining representation of the projection of $\cdot$ into $\mathcal{A}_{\alpha}$ |
| $N_{\alpha}$ | Dimension of vector space on which the defining representation of $\mathcal{A}$ acts |
| $N$ | Sum of $N_{\alpha}$ |
| $\Tr_{\alpha}\left(\cdot\right)$ | Trace of $\cdot$ in the defining representation of $\mathcal{A}_{\alpha}$ |
| $\Tr\left(\cdot\right)$ | Trace of $\cdot$ according to its representation in $\mathcal{H}^{N}\left(\mathbb{C}\right)\supseteq\mathcal{A}$ |
| $\left\lVert\cdot\right\rVert_{\text{op}}$ | Operator norm of $\cdot\in\mathcal{A}_{\alpha}$ in its defining representation |
| $\left\lVert\cdot\right\rVert_{\ast}$ | Trace (nuclear) norm of $\cdot\in\mathcal{A}_{\alpha}$ in its defining representation |
| $\left\lVert\cdot\right\rVert_{\textrm{F}}$ | Frobenius norm of $\cdot\in\mathcal{A}_{\alpha}$ in its defining representation |
| $\operatorname{O}\left(\operatorname{poly}\left(N\right)\right)$ | $\operatorname{O}\left(N^{\kappa}\right)$ for some constant $\kappa>0$ |
| WLOG | without loss of generality |
| w.h.p. | with high probability |

### A.1 Quantum Neural Networks

We first review *quantum neural networks* (QNNs). These are defined by a parameterized *ansatz*

$$ $\bm{U}\left(\bm{\theta}\right)=\prod_{i=p}^{1}\exp\left(-\mathrm{i}\theta_{i} \bm{A}_{i}\right),$ (39) $$

which belongs to some path-connected subgroup of the unitary group $\operatorname{U}\left(N\right)$. Though $\bm{U}\left(\bm{\theta}\right)$ can in principle parameterize all of $\operatorname{U}\left(N\right)$, it is often taken to instead parameterize some path-connected subgroup $G\subseteq\operatorname{U}\left(N\right)$. This might be due to enforcing some global structure on the model [], or can be a model of the finiteness of the reverse light cones of shallow quantum circuits [].

Given a choice of ansatz, the goal of a QNN is to minimize an empirical risk of the form:

$$ $f\left(\bm{\theta}\right)=\frac{1}{\left\lvert\mathcal{R}\right\rvert}\sum_{ \bm{\rho}\in\mathcal{R}}\ell\left(\bm{\theta};\bm{\rho}\right)=\frac{1}{\left \lvert\mathcal{R}\right\rvert}\sum_{\bm{\rho}\in\mathcal{R}}\Tr\left(\bm{U} \left(\bm{\theta}\right)\bm{\rho}\bm{U}\left(\bm{\theta}\right)^{\dagger}\bm{O }\right).$ (40) $$

Here, $\mathcal{R}$ can be thought of a data set comprising multiple input states $\bm{\rho}$, and $\ell$ the loss function. Historically, when $\left\lvert\mathcal{R}\right\rvert=1$ QNNs have been referred to as *variational quantum algorithms* (VQAs) [] due to their connection to finding variational approximations to the ground states of quantum Hamiltonians. There are known quantum-classical separations for the expressivity of quantum neural networks even when taking into account the requirement that the training procedure is efficient [, , ], though they require very specific training algorithms that take advantage of the structure of the data.

There has been recent hope that, with enough ansatz structure, efficient training may follow just via a simple application of gradient descent. This follows from the *Lie algebra-supported ansatz* (LASA) literature, where it has been shown that if the generators $\mathrm{i}\bm{A}_{i}$ of the ansatz $\bm{U}$ as well as the (scaled) objective observable $\mathrm{i}\bm{O}$ belong to the same Lie algebra $\mathfrak{g}$—called the *dynamical Lie algebra* []—generating $G$, gradients scale inversely with the dimension of $\mathfrak{g}$ [, ]. It is also conjectured that when $\dim\left(\mathfrak{g}\right)$ scales polynomially with the system size there exist polynomial-depth ansatzes that do not have poor local minima, which together with the large gradients would imply efficient trainability of these loss functions via gradient descent. Conditioned on this conjecture there have been results demonstrating expressivity separations in quantum machine learning where the QNN is efficiently trainable through a simple application of gradient descent [, ].

Though the LASA framework gives sufficient conditions for loss functions to have large gradients, it is known that they are not necessary. One example of this was demonstrated in Diaz *et al.* [], where it was shown that parameterized matchgate circuits with an objective observable given by constant-degree polynomials in Majorana fermions are efficiently trainable though they are not a part of the LASA setting. We claim that both settings are special cases of a Jordan algebraic understanding of variational loss functions. In preparation of discussing this connection we now review Jordan algebras.

### A.2 Jordan Algebras

A *Jordan algebra* over the reals is formally a real vector space $V$ with a commutative multiplication operation $\circ$ acting on $u,v\in V$ satisfying the *Jordan identity*:

$$ $u\circ\left(\left(u\circ u\right)\circ v\right)=\left(u\circ u\right)\circ \left(u\circ v\right),$ (41) $$

which ensures the associativity of the power. A simple example of a Jordan algebra over the reals is the real algebra $\mathcal{H}^{N}\left(\mathbb{C}\right)$ of $N\times N$ complex-valued Hermitian matrices with $\circ$ given by half the anticommutator. In particular, for any finite-dimensional quantum system both $\bm{O}$ and the inputs $\bm{\rho}$ in Eq. (40) belong to $\mathcal{H}^{N}\left(\mathbb{C}\right)$. We emphasize that though this algebra is typically written in terms of complex matrices it is still a *real* Jordan algebra. This is because, for instance, $\mathrm{i}\not\in\mathbb{R}$ multiplying a Hermitian matrix is no longer Hermitian. In fact, the Jordan algebra of Hermitian matrices is *Euclidean* (or *formally real*) as it satisfies the defining property []:

$$ $\forall A,B\in\mathcal{A},\,A\circ A+B\circ B=0\iff A=B=0.$ (42) $$

It is apparent that all subalgebras of a formally real algebra are also formally real.

The real vector space $V$ with which a Jordan algebra is associated also has a natural linear transformation $L\left(u\right)$:

$$ $L\left(u\right)v\equiv u\circ v,$ (43) $$

which can be viewed as the Jordan algebraic analogue of the adjoint representation of Lie algebras. This linear transformation gives rise to the *canonical trace form*:

$$ $\tau\left(u,v\right)\equiv\Tr\left(L\left(u\circ v\right)\right).$ (44) $$

When $\tau$ is nonsingular we call the associated Jordan algebra $\mathcal{A}$ *semisimple*. As an example, for the algebra $\mathcal{H}^{N}\left(\mathbb{F}\right)$ of Hermitian matrices over the field $\mathbb{F}$, $\tau\left(u,v\right)$ is just the Frobenius inner product between $u$ and $v$. Just as is the case for semisimple Lie algebras, all symmetric bilinear forms on a semisimple Jordan algebra are identical up to an overall scaling. Specifically, for any symmetric bilinear form $\sigma$, there exists an element $z$ in the center of $\mathcal{A}$ such that (Theorem 10, Koecher []):

$$ $\sigma\left(u,v\right)=\tau\left(z\circ u,v\right).$ (45) $$

For $\mathcal{A}=\mathcal{H}^{N}\left(\mathbb{C}\right)$ in the defining representation the center is just real multiples of the identity matrix, i.e., there exists a real $I\geq 0$ such that:

$$ $I\sigma\left(u,v\right)=\tau\left(u,v\right).$ (46) $$

We call $I$ the *index* of $\sigma$ in analogy with Fontana *et al.* [] for Lie algebras.

Though perhaps not as famous as the classification of compact Lie groups, the semisimple Euclidean Jordan algebras have also been classified.

###### Theorem 8 (Classification of semisimple Euclidean Jordan algebras [ 32 ] ) .

Any semisimple Euclidean Jordan algebra is isomorphic to a direct sum of the simple Euclidean Jordan algebras:

- •
$\mathcal{L}_{N}$ for $N\geq 1$: the *spin factor*, with vector space equal to $\mathbb{R}^{N}$ and $\circ$ given by the operation
$\bm{x}\circ\bm{y}=\left(\bm{x}\cdot\bm{y},x_{1}\bm{\overline{y}}+y_{1}\bm{
\overline{x}}\right),$
(47)
where $\overline{\cdot}$ denotes $\cdot$ with the first coordinate projected out;
- •
$\mathcal{H}^{N}\left(\mathbb{R}\right)$ for $N\geq 3$: $N\times N$ symmetric matrices over $\mathbb{R}$, with $\circ$ half the anticommutator;
- •
$\mathcal{H}^{N}\left(\mathbb{C}\right)$ for $N\geq 3$: $N\times N$ Hermitian matrices over $\mathbb{C}$, with $\circ$ half the anticommutator;
- •
$\mathcal{H}^{N}\left(\mathbb{H}\right)$ for $N\geq 3$: $N\times N$ Hermitian matrices over $\mathbb{H}$, with $\circ$ half the anticommutator;
- •
$\mathcal{H}^{3}\left(\mathbb{O}\right)$: $3\times 3$ Hermitian matrices over $\mathbb{O}$, with $\circ$ half the anticommutator.

We use the term *defining representation* to speak of the described matrix representations of these algebras. As the Hermitian octonion case is not an infinite family it is often called *exceptional*. We here are interested in asymptotic sequences of Jordan algebras so we will not be considering the exceptional case.

Every Jordan algebra $\mathcal{A}$ has associated with it an automorphism group $\operatorname{Aut}\left(\mathcal{A}\right)$. As an example, for $\mathcal{H}^{N}\left(\mathbb{R}\right)$ in the defining representation this is just given by the action of conjugation (under the usual matrix multiplication) by orthogonal matrices. More generally, the nonexceptional simple cases have automorphism groups []:

- •
$\operatorname{Aut}\left(\mathcal{L}_{N}\right)$: left-action of $\left\{1\right\}\times\operatorname{O}\left(N-1\right)$;
- •
$\operatorname{Aut}\left(\mathcal{H}^{N}\left(\mathbb{R}\right)\right)$: conjugation action of $\operatorname{PO}\left(N\right)$;
- •
$\operatorname{Aut}\left(\mathcal{H}^{N}\left(\mathbb{C}\right)\right)$: disjoint union of the conjugation action of $\operatorname{PU}\left(N\right)$, and the transpose followed by the conjugation action of $\operatorname{PU}\left(N\right)$;
- •
$\operatorname{Aut}\left(\mathcal{H}^{N}\left(\mathbb{H}\right)\right)$: conjugation action of $\operatorname{PSp}\left(N\right)$.

We are here primarily interested in the connected component $\operatorname{Aut}_{1}\left(\cdot\right)\subseteq\operatorname{Aut}\left(\cdot\right)$ containing the identity transformation, so we will only concern ourselves with the path-connected automorphism subgroups:

- •
$\operatorname{SO}\left(N-1\right)\cong\operatorname{Aut}_{1}\left(\mathcal{L}_
{N}\right)\subset\operatorname{Aut}\left(\mathcal{L}_{N}\right)$;
- •
$\operatorname{SO}\left(N\right)\cong\operatorname{Aut}_{1}\left(\mathcal{H}^{N
}\left(\mathbb{R}\right)\right)\subseteq\operatorname{Aut}\left(\mathcal{H}^{N
}\left(\mathbb{R}\right)\right)$;
- •
$\operatorname{SU}\left(N\right)\cong\operatorname{Aut}_{1}\left(\mathcal{H}^{N
}\left(\mathbb{C}\right)\right)\subset\operatorname{Aut}\left(\mathcal{H}^{N}
\left(\mathbb{C}\right)\right)$;
- •
$\operatorname{Sp}\left(N\right)\cong\operatorname{Aut}_{1}\left(\mathcal{H}^{N
}\left(\mathbb{H}\right)\right)\subset\operatorname{Aut}\left(\mathcal{H}^{N}
\left(\mathbb{H}\right)\right)$.

Surprisingly, the automorphism groups of semisimple Jordan algebras are also classified. Given a decomposition of a semisimple Euclidean Jordan algebra $\mathcal{A}$ into simple components:

$$ $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha},$ (48) $$

the connected component of $\operatorname{Aut}_{1}\left(\mathcal{A}\right)\subseteq\operatorname{Aut}\left
(\mathcal{A}\right)$ containing the identity is isomorphic to the direct product (Theorem 10, Koecher []):

$$ $\operatorname{Aut}_{1}\left(\mathcal{A}\right)\cong\bigtimes_{\alpha} \operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right).$ (49) $$

###### Theorem 8 (Classification of semisimple Euclidean Jordan algebras [ 32 ] ) .

### A.3 ϵ italic-ϵ \epsilon italic_ϵ -Approximate t 𝑡 t italic_t -Designs Over Aut 1 ⁡ ( 𝒜 ) subscript Aut 1 𝒜 \operatorname{Aut}_{1}\left(\mathcal{A}\right) roman_Aut start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ( caligraphic_A )

In order to discuss the structure of the loss function in any detail we will need to consider some choice of randomness over loss functions. To achieve this we will use *$\epsilon$-approximate $t$-designs* over $\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. Our use of these designs formalizes the notion of approximate “independence” or “uniform initialization” of the ansatz with respect to the eigenbasis of a given objective observable, as when $\epsilon\to 0$ and $t\to\infty$ the ansatz is chosen in a completely group-invariant way over $\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. By Eq. (49) these designs are (up to isomorphism) direct products of $\epsilon$-approximate $t$-designs over $\operatorname{SO}\left(N\right)$, $\operatorname{SU}\left(N\right)$, and $\operatorname{Sp}\left(N\right)$.

Before defining $\epsilon$-approximate $t$-designs we define the Haar measure on compact Lie groups such as $\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. This is the unique group-invariant normalized measure on these compact Lie groups.

###### Definition 9 (Haar measure on $G$ [ 45 ] ) .

Let $G$ be a compact Lie group. The unique measure satisfying:

$$ $\int_{G}\differential{\mu}=1$ (50) $$

as well as

$$ $\differential{\mu}=\differential{\left(g\mu\right)}$ (51) $$

for all $g\in G$ is the *Haar measure on $G$*. The existence and uniqueness of this measure follows from Haar [].

We now define $\epsilon$-approximate $t$-designs. We will here use the “trace norm definition” out of convenience, though all of the commonly used definitions are roughly equivalent [].

###### Definition 10 ( $\epsilon$ -approximate $t$ -designs over $G$ ) .

Let $G$ be a compact Lie group with defining representation over an $N$-dimensional space, and $\mu$ the Haar measure over $G$. A measure $\nu$ satisfying:

$$ $\sum_{m}\left\lvert\mathbb{E}_{U\sim\mu}\left[m\left(U\right)\right]-\mathbb{E }_{U\sim\nu}\left[m\left(U\right)\right]\right\rvert\leq\epsilon,$ (52) $$

where the sum is over all degree-$\left(t,t\right)$ monomials $m\left(U\right)$ in the entries of $U$ and $U^{\ast}$, is an *$\epsilon$-approximate $t$-design over $G$*.

###### Definition 9 (Haar measure on G 𝐺 G italic_G [ 45 ] ) .

###### Definition 10 ( ϵ italic-ϵ \epsilon italic_ϵ -approximate t 𝑡 t italic_t -designs over G 𝐺 G italic_G ) .

### A.4 Wishart Matrices

Our main results will be given in terms of Wishart-distributed random matrices, so before proceeding we give a brief review of this distribution. We will use $\mathcal{W}_{n}^{\beta}\left(r,\bm{\Sigma}\right)$ to denote the *$\beta$-Wishart distribution* [], where:

- •
$\beta=1,2,4$ indicates it is over a field $\mathbb{R},\mathbb{C},\mathbb{H}$, respectively;
- •
$r\in\mathbb{N}_{1}$ is the *degrees of freedom* parameter of the distribution;
- •
$\bm{\varSigma}\in\mathcal{H}^{n}\left(\mathbb{R}\right)$ is the symmetric, real-valued, positive-definite *scale matrix* parameter of the distribution.

When we do not specify $\beta$ we implicitly are referring to the usual Wishart distribution where $\beta=1$. The $\beta$-Wishart distribution is over positive semidefinite matrices of rank $r$, constructed by drawing i.i.d. standard Gaussian entries over $\mathbb{F}$:

$$ $X_{i,j}\sim\mathcal{N}^{\beta}\left(0,1\right)$ (53) $$

for $i\in\left[n\right]$ and $j\in\left[r\right]$, and considering the distribution of:

$$ $\bm{W}=\sqrt{\bm{\varSigma}}\bm{X}\bm{X}^{\dagger}\sqrt{\bm{\varSigma}}.$ (54) $$

Here there exists the so-called *Bartlett decomposition* for $\bm{W}\sim\mathcal{W}_{n}^{\beta}\left(r,\bm{I}_{n}\right)$:

$$ $\bm{W}\sim\sqrt{\bm{\varSigma}}\bm{L}\bm{L}^{\dagger}\sqrt{\bm{\varSigma}},$ (55) $$

where $\bm{L}$ is lower-triangular. When $r\geq n$, the the entries below the diagonal of $\bm{L}$ are i.i.d. $\mathcal{N}^{\beta}\left(0,1\right)$-distributed and the diagonal entries are i.i.d. distributed as []:

$$ $\beta L_{i,i}^{2}\sim\chi^{2}\left(\beta\left(r-i+1\right)\right),$ (56) $$

i.e., the $L_{i,i}$ are $\chi$-distributed up to an overall scaling of $\beta^{-\frac{1}{2}}$. When $r<n$, $\bm{L}$ is $n\times r$ with first $r$ rows as above and all other entries i.i.d. according to $\mathcal{N}^{\beta}\left(0,1\right)$ [, , ]. This completely characterizes the marginal distribution of the entries of any $\bm{W}\sim\mathcal{W}_{n}^{\beta}\left(r,\bm{I}_{n}\right)$.

Our final results will be written in terms of real Wishart matrices $\bm{W}$, i.e., those with $\beta=1$. When we analyze the asymptotic density of local minima of a JAWS it will turn out we will need to consider the asymptotic spectrum of a real Wishart matrix, which we now discuss. When the scale matrix $\bm{\varSigma}$ of $\bm{W}$ is the identity, as $n,r\to\infty$ with $\frac{n}{r}=\gamma$ held constant, the spectrum of $r^{-1}\bm{W}$ is known to almost surely converge weakly to a fixed distribution []. This fixed distribution is the *Marčenko–Pastur distribution*, given by

$$ $\derivative{\mu_{\text{MP}}^{\gamma}}{\lambda}=\left(1-\gamma^{-1}\right) \delta\left(\lambda\right)\bm{1}\left\{\gamma\geq 1\right\}+\frac{\sqrt{\left( \gamma_{+}-\lambda\right)\left(\lambda-\gamma_{-}\right)}}{2\uppi\gamma\lambda }\bm{1}\left\{\gamma_{-}\leq\lambda\leq\gamma_{+}\right\},$ (57) $$

where

$$ $\gamma_{\pm}\equiv\left(1\pm\sqrt{\gamma}\right)^{2}$ (58) $$

and $\bm{1}$ is the indicator function. This distribution is illustrated in Fig. 4.

Figure: Figure 4: Marčenko–Pastur distribution. The density of the Marčenko–Pastur distribution—the asymptotic empirical eigenvalue distribution of normalized Wishart matrices—in the regime where *$\gamma\ll 1$* and where *$\gamma=1$*. At $\gamma=1$ the associated Wishart matrix transitions from being full-rank to being low-rank.
Refer to caption: x4.png

### A.5 Convergence of Random Variables

We finally discuss in more detail the various of notions of convergence of random variables that we use throughout our paper. Given a sequence of a set $\left(X_{i}^{N}\right)_{i}$ of real-valued random variables as $N\to\infty$, we say $\left(X_{i}^{N}\right)_{i}$ *weakly converges* (or *converges in distribution*) to $\left(X_{i}^{\infty}\right)_{i}$ if the joint cumulative distribution function $F^{N}$ of the $X_{i}^{N}$ converges to the joint cumulative distribution function $F^{\infty}$ of the $X_{i}^{\infty}$ at every point at which $F^{\infty}$ is continuous. That is,

$$ $\lim_{N\to\infty}F^{N}\left(\bm{x}\right)=F^{\infty}\left(\bm{x}\right)$ (59) $$

for all points $\bm{x}$ at which $F^{\infty}$ is continuous. We denote this at the level of random variables with the notation:

$$ $\left(X_{i}^{N}\right)_{i}\rightsquigarrow\left(X_{i}^{\infty}\right)_{i}.$ (60) $$

One way to quantify the rate of this convergence is through the use of the *Lévy–Prokhorov metric*. Letting $p^{N}$ and $p^{\infty}$ be the densities associated with $F^{N}$ and $F^{\infty}$, respectively, the Lévy–Prokhorov metric is given by:

$$ $\pi\left(p^{N},p^{\infty}\right)=\inf\left\{\epsilon>0\mid\forall A,\,F^{N} \left(A\right)\leq F^{\infty}\left(A^{\epsilon}\right)+\epsilon\right\},$ (61) $$

where here $A^{\epsilon}$ is an $\epsilon$-neighborhood in $\infty$-norm of $A$. $\pi\left(p^{N},p^{\infty}\right)\to 0$ if and only if $\left(X_{i}^{N}\right)_{i}\rightsquigarrow\left(X_{i}^{\infty}\right)_{i}$.

Convergence in distribution is closely related to the pointwise convergence of probability densities. Indeed, the latter implies the former by Scheffé’s theorem []. The former implies the latter if the $p^{N}$ are equicontinuous and uniformly bounded as $N\to\infty$ [].

We finally discuss *convergence in probability*. This is the statement that, for all $\epsilon>0$,

$$ $\lim_{N\to\infty}\mathbb{P}\left(\left\lVert\bm{X}^{N}-\bm{X}^{\infty}\right \rVert_{\infty}>\epsilon\right)=0.$ (62) $$

We denote this convergence using the notation:

$$ $\left(X_{i}^{N}\right)_{i}\xrightarrow{p}\left(X_{i}^{\infty}\right)_{i}.$ (63) $$

One way to quantify convergence in probability is through the *Ky Fan metric*, which is given by:

$$ $\alpha\left(p^{N},p^{\infty}\right)=\inf\left\{\epsilon>0\mid\mathbb{P}\left( \left\lVert\bm{X}^{N}-\bm{X}^{\infty}\right\rVert_{\infty}>\epsilon\right)\leq \epsilon\right\}.$ (64) $$

$\alpha\left(p^{N},p^{\infty}\right)\to 0$ if and only if $\left(X_{i}^{N}\right)_{i}\xrightarrow{p}\left(X_{i}^{\infty}\right)_{i}$. The Ky Fan metric upper bounds the Lévy–Prokhorov metric [], consistent with the notion of convergence in probability implying convergence in distribution.

We will in what follows often be loose with language and say two sequences of distributions converge to one another at a rate $K$, either weakly or in probability, rather than state convergence to a fixed distribution. This should be understood to mean that their distance in associated metric asymptotically vanishes as $K^{-1}$.

## Appendix B Formal Discussion of the Main Results

### B.1 The Jordan Algebraic Structure of Variational Loss Landscapes

With these preliminaries in place we can write the variational loss function given in Eq. (40) as the canonical trace form on $\mathcal{H}^{N}\left(\mathbb{C}\right)$:

$$ $\ell\left(\bm{\theta};\bm{\rho}\right)=\tau\left(\bm{\rho},T\left(\bm{\theta} \right)\bm{O}\right),$ (65) $$

where here $\bm{\rho},\bm{O}\in\mathcal{H}^{N}\left(\mathbb{C}\right)$, and $T\left(\bm{\theta}\right)$ parameterizes some path-connected compact Lie group $\mathcal{T}$. Note that $\left\{T\left(\bm{\theta}\right)\bm{O}\right\}_{\bm{\theta}}$ generates a Jordan subalgebra $\mathcal{A}\subseteq\mathcal{H}^{N}\left(\mathbb{C}\right)$ such that $\bm{O}\in\mathcal{A}$ and $\mathcal{T}\subseteq\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. As $T\left(\bm{\theta}\right)\bm{O}\in\mathcal{A}$ for all $\bm{\theta}$, the trace form is zero for components of $\bm{\rho}$ orthogonal to $\mathcal{A}$. Because of this we will often also consider $\bm{\rho}$ an element of $\mathcal{A}$ in a slight abuse of notation.

We call a variational loss function of this form a *Jordan algebra-supported ansatz* (JASA), in analogy with the term *Lie-algebra supported ansatz* (LASA) introduced in Fontana *et al.* []. However, whereas variational loss functions are LASAs only if $\mathrm{i}\bm{O}$ belongs to the dynamical Lie algebra generating the ansatz, *all* variational loss functions are JASAs (up to assuming the path-connectedness of $\mathcal{T}$). Indeed, in Appendix G we give a direct mapping from both LASAs and the variational matchgate formalism of Diaz *et al.* [] to JASAs and show that a mapping in the other direction is generally not possible.

We are often interested in the case where our variational loss landscape has some sort of structure, i.e., when $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha}\neq\mathcal{H}^{N}\left
(\mathbb{C}\right)$. In Appendix F we demonstrate that in the context of examining variational loss landscapes, the spin factor sectors ($\mathcal{A}_{\alpha}=\mathcal{L}_{N_{\alpha}}$) effectively reduce to real symmetric ($\mathcal{A}_{\alpha}=\mathcal{H}^{N_{\alpha}}\left(\mathbb{R}\right)$) sectors. Because of this, we will from here on out focus on the case when $\mathcal{A}_{\alpha}$ is the Jordan algebra of Hermitian matrices over a field, i.e., $\mathcal{A}_{\alpha}\cong\mathcal{H}^{N_{\alpha}}\left(\mathbb{F}_{\alpha}\right)$ for $\mathbb{F}_{\alpha}=\mathbb{R},\mathbb{C},\mathbb{H}$. We will also throughout use $\beta_{\alpha}=1,2,4$ to label these three cases, respectively, as is commonly done in the physics literature. We use $\Tr_{\alpha}\left(\cdot\right)$ to denote the trace of $\cdot$ in the defining representation of $\mathcal{A}_{\alpha}$ and $\Tr\left(\cdot\right)$ to denote the trace of $\cdot$ according to its representation in $\mathcal{H}^{N}\left(\mathbb{C}\right)\supseteq\mathcal{A}$.

Recalling from Appendix A.2 the universality of the trace form and the direct product decomposition of $\operatorname{Aut}_{1}\left(\mathcal{A}\right)$, we can rewrite the loss function of Eq. (65) as:

$$ $\ell\left(\bm{\theta};\bm{\rho}\right)=\sum_{\alpha}\ell^{\alpha}\left(\bm{ \theta};\bm{\rho}\right)\equiv\sum_{\alpha}I_{\alpha}\tau^{\alpha}\left(\bm{ \rho}^{\alpha},T^{\alpha}\left(\bm{\theta}\right)\bm{O}^{\alpha}\right),$ (66) $$

where here $\cdot^{\alpha}$ is used to denote the component of $\cdot$ in $\mathcal{A}_{\alpha}$, $\tau^{\alpha}$ is the canonical trace form on $\mathcal{A}_{\alpha}$, and $I_{\alpha}>0$ is the constant due to Eq. (46).

We now give an explicit form for $T^{\alpha}\left(\bm{\theta}\right)$. Recall that elements of $\operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right)$ are, in the defining representation of $\mathcal{A}_{\alpha}$, given by the conjugation action by elements of $\operatorname{SO}\left(N_{\alpha}\right)$, $\operatorname{SU}\left(N_{\alpha}\right)$, or $\operatorname{Sp}\left(N_{\alpha}\right)$ when $\mathbb{F}_{\alpha}=\mathbb{R},\mathbb{C},\mathbb{H}$, respectively. We let $G_{\alpha}$ denote the corresponding Lie group and $\mathfrak{g}_{\alpha}$ the associated Lie algebra. We also denote $G$ as the direct product over $G_{\alpha}$, and $\mathfrak{g}$ the corresponding Lie algebra. Following the typical structure of variational quantum algorithms [] we will assume $T^{\alpha}\left(\bm{\theta}\right)$ in the defining representation corresponds to conjugation by:

$$ $U_{\bm{g},h}^{\alpha}\left(\bm{\theta}\right)=g_{0}^{\alpha\dagger}\left(\prod _{i=1}^{p}g_{i}^{\alpha}\exp\left(\theta_{i}A_{i}^{\alpha}\right)g_{i}^{\alpha \dagger}\right)h^{\alpha}\in G_{\alpha}$ (67) $$

for some $g_{i},h\in G$ and $A_{i}\in\mathfrak{g}$, where $\cdot^{\alpha}$ denotes the projection onto $G_{\alpha}$ or $\mathfrak{g}_{\alpha}$ appropriately. With this choice of ansatz it is also easy to see the derivatives of $\ell$ have algebraic interpretations. For instance, at $\bm{\theta}=\bm{0}$, $i\geq j$,

$$ $\displaystyle\partial_{i}\ell\left(\bm{\theta};\bm{\rho}\right)$ $\displaystyle=\sum_{\alpha}I_{\alpha}\tau^{\alpha}\left(g_{0}^{\alpha}\bm{\rho }^{\alpha}g_{0}^{\alpha\dagger},\left[g_{i}^{\alpha}A_{i}^{\alpha}g_{i}^{ \alpha\dagger},h^{\alpha}\bm{O}^{\alpha}h^{\alpha\dagger}\right]\right),$ (68) $\displaystyle\partial_{i}\partial_{j}\ell\left(\bm{\theta};\bm{\rho}\right)$ $\displaystyle=\sum_{\alpha}I_{\alpha}\tau^{\alpha}\left(g_{0}^{\alpha}\bm{\rho }^{\alpha}g_{0}^{\alpha\dagger},\left[g_{j}^{\alpha}A_{j}^{\alpha}g_{j}^{ \alpha\dagger},\left[g_{i}^{\alpha}A_{i}^{\alpha}g_{i}^{\alpha\dagger},h^{ \alpha}\bm{O}^{\alpha}h^{\alpha\dagger}\right]\right]\right),$ (69) $$

where the commutator action of the automorphism group of a Jordan algebra is canonically defined through its representation $L$ defined in Eq. (43) (see Lemma 7 of Koecher []).

### B.2 Jordan Algebraic Wishart Systems

We have demonstrated that all variational loss landscapes are Jordan algebra-supported ansatzes (JASAs). We are finally ready to discuss our main results, which give an explicit expression for the loss landscape of a JASA when its ansatz takes the form of Eq. (67). To do this we first define a *Jordan algebraic Wishart system* (JAWS), leaving for now ambiguous the connection to Wishart matrices.

###### Definition 11 (Jordan algebraic Wishart system) .

Let $O$ belong to a Jordan algebra $\mathcal{A}\subseteq\mathcal{H}^{N}\left(\mathbb{C}\right)$ and let $\mathcal{T}$ be a path-connected subspace $\mathcal{T}\subseteq\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. Let

$$ $\mathcal{A}\cong\bigoplus_{\alpha}\mathcal{A}_{\alpha}$ (70) $$

be the decomposition of $\mathcal{A}$ into simple Euclidean components as in Theorem 8 with associated decomposition $\mathcal{T}\cong\bigtimes_{\alpha}\mathcal{T}_{\alpha}$ as in Eq. (49). Let $\mathcal{G}_{i}^{\alpha},\mathcal{H}^{\alpha}$ be independent distributions over $\mathcal{T}_{\alpha}$, and $\bm{A}=\left\{A_{i}\right\}_{i=1}^{p}$ elements of the Lie algebra of $\operatorname{Aut}_{1}\left(\mathcal{A}\right)$. We call

$$ $\mathcal{J}=\left(\mathcal{A},\mathcal{T},\bm{\mathcal{G}},\bm{\mathcal{H}}, \bm{A},O\right)$ (71) $$

a *Jordan algebraic Wishart system* (JAWS).

Every JAWS has associated with it a loss function as discussed in Appendix B.1. Using $\cdot^{\alpha}$ to label the projection of $\cdot$ into $\mathcal{A}_{\alpha}$ in its defining representation, the associated loss function takes the form:

$$ $\ell\left(\bm{\theta};\bm{\rho}\right)=\sum_{\alpha}\ell^{\alpha}\left(\bm{ \theta};\bm{\rho}\right)=\sum_{\alpha}I_{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{ \alpha}\bm{U}^{\alpha}\left(\bm{\theta}\right)\bm{O}^{\alpha}\bm{U}^{\alpha} \left(\bm{\theta}\right)^{\dagger}\right),$ (72) $$

where $\bm{U}^{\alpha}\left(\bm{\theta}\right)\in G_{\alpha}$ for all $\bm{\theta}$ and is of the form:

$$ $\bm{U}^{\alpha}\left(\bm{\theta}\right)\equiv g_{0}^{\alpha\dagger}\left(\prod _{i=1}^{p}g_{i}^{\alpha}\exp\left(\theta_{i}A_{i}^{\alpha}\right)g_{i}^{\alpha \dagger}\right)h^{\alpha}.$ (73) $$

Here, $g_{i}^{\alpha}$ is the defining representation of an element drawn from from the distribution $\mathcal{G}_{i}^{\alpha}$, and $h^{\alpha}$ is similar for $\mathcal{H}^{\alpha}$.

Our main result is a concise, asymptotic description of $\ell\left(\bm{\theta};\bm{\rho}\right)$ when the ansatz is chosen “sufficiently independently” from $\bm{O}$ (up to respecting the algebra $\mathcal{A}$). More formally, the rate at which $\ell\left(\bm{\theta};\bm{\rho}\right)$ converges to its asymptotic limit depends on the parameters defined by the following assumption.

###### Assumption 12 ( $\epsilon$ -approximate $t$ -design of observable basis) .

Each $\mathcal{H}^{\alpha}$ is an $\epsilon$-approximate $t$-design over $\mathcal{T}_{\alpha}=\operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right)$.

If there is a sense of geometric locality in the system, i.e., if the reverse lightcone of $\bm{O}^{\alpha}$ is of the form:

$$ $\bm{U}^{\alpha}\left(\bm{\theta}\right)\bm{O}^{\alpha}\bm{U}^{\alpha}\left(\bm {\theta}\right)^{\dagger}=\bm{\tilde{O}}^{\alpha}\otimes\bm{I}$ (74) $$

for some $\bm{\tilde{O}}^{\alpha}$ acting on a space of dimension at most $D\equiv\exp\left(\operatorname{O}\left(p^{d}\right)\right)$ for constant geometric dimension $d$, one can explicitly enforce this with only a modest overhead in circuit depth. Indeed, one may embed $\mathcal{A}_{\alpha}$ into qubits and use the construction of Harrow and Mehraban [] to give a $d$-dimensional random circuit of depth $\operatorname{O}\left(\operatorname{poly}\left(t\right)p\right)$ that achieves this over $\operatorname{Aut}_{1}\left(\mathcal{H}^{2^{n}}\left(\mathbb{C}\right)\right)
\supset\operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right)$ for some $n=\operatorname{O}\left(p^{d}\right)$ and $\epsilon=\exp\left(-\operatorname{\Omega}\left(p\right)\right)$.

Given Assumption 12 we are able to prove our main result on the convergence of loss functions. We are able to prove our convergence for any $N$-dependent overall normalization of the loss function $\mathcal{N}$, the only requirement being that central moments of sufficiently large (constant) order have a finite limit as $N\to\infty$. This is always true for $\mathcal{N}=1$ and, depending on the specific choice of $t$-design, potentially holds for any $\mathcal{N}\leq\operatorname{O}\left(N^{1-\delta}\right)$ where $\delta>0$ is constant.

###### Theorem 13 (Loss function distribution) .

Let $\mathcal{J}$ be a JAWS satisfying Assumption 12 with loss function $\ell\left(\bm{\theta};\bm{\rho}\right)$ for $\bm{\rho}\in\mathcal{R}$. Further assume that

$$ $\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)=\operatorname{\Omega}\left (\frac{1}{N_{\alpha}^{0.999}}\right).$ (75) $$

Fix $\bm{\theta}$. Assume $\mathcal{N}\leq\operatorname{O}\left(\min_{\alpha}N_{\alpha}^{0.99}\right)$ is such that there exists a constant $k^{\ast}$ such that the central moments of order $k>k^{\ast}$ of $\left(\mathcal{N}\ell\left(\bm{\theta};\bm{\rho}\right)\right)_{\bm{\rho}\in
\mathcal{R}}$ are finite as all $\epsilon^{-1},t,N_{\alpha}\to\infty$. We have the convergence in joint distributions:

$$ $\left(\mathcal{N}\ell\left(\bm{\theta};\bm{\rho}\right)\right)_{\bm{\rho}\in \mathcal{R}}\rightsquigarrow\left(\mathcal{N}\hat{\ell}\left(\bm{\theta};\bm{ \rho}\right)\right)_{\bm{\rho}\in\mathcal{R}}$ (76) $$

as all $\epsilon^{-1},t,N_{\alpha}\to\infty$, where

$$ $\hat{\ell}\left(\bm{\theta};\bm{\rho}\right)\equiv\sum_{\alpha}\hat{\ell}^{ \alpha}\left(\bm{\theta};\bm{\rho}\right)\equiv\sum_{\alpha}\frac{I_{\alpha} \overline{o}^{\alpha}}{r_{\alpha}}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\bm{W}^{ \alpha}\right),$ (77) $$

$\overline{o}^{\alpha}$ is the arithmetic mean eigenvalue of $\bm{O}^{\alpha}$ in the defining representation:

$$ $\overline{o}^{\alpha}=N_{\alpha}^{-1}\Tr_{\alpha}\left(\bm{O}^{\alpha}\right),$ (78) $$

and the $\bm{W}^{\alpha}$ are independent Wishart-distributed random matrices with $\beta_{\alpha}r_{\alpha}$ degrees of freedom, where

$$ $r_{\alpha}\equiv\frac{\left\lVert\bm{O}^{\alpha}\right\rVert_{\ast}^{2}}{\left \lVert\bm{O}^{\alpha}\right\rVert_{\textrm{F}}^{2}}.$ (79) $$

In particular, the distributions differ in Lévy–Prokhorov metric by

$$ $\pi=\operatorname{O}\left(\frac{\log\log\left(\mathcal{N}^{-t}\epsilon^{-1} \right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+\frac{\log\left(t \right)}{\sqrt{t}}+\sum_{\alpha}\sqrt{\frac{\mathcal{N}\log\left(N_{\alpha} \right)}{N_{\alpha}^{1.001}}}\right)=\operatorname{o}\left(1\right).$ (80) $$

In Appendix H we show that Eq. (75) can effectively be taken WLOG. This is because all $\bm{\rho}^{\alpha}$ for which this is not true are such that $\ell^{\alpha}\left(\bm{\theta};\bm{\rho}\right)$ have asymptotically vanishing variance even when rescaled by the potentially exponentially large $\mathcal{N}=\operatorname{O}\left(N\right)$. This is why we do not take Eq. (75) as a numbered Assumption. Examples of the loss density are illustrated in Fig. 2(a).

We can strengthen the statement of weak convergence in Theorem 13 to one of pointwise convergence of densities given equicontinuity and boundedness of the appropriately normalized loss function density. Whether this is true depends on the specifics of the distributions $\bm{\mathcal{G}},\bm{\mathcal{H}}$, particularly whether the loss is equicontinuous and bounded at the same $\mathcal{N}=\sqrt{r_{\alpha}}$ scale as $\hat{\ell}^{\alpha}$. We give some standard examples where this is true in Appendix J.

###### Corollary 14 (Convergence of loss function densities) .

Let $\mathcal{J},\ell,\hat{\ell}$ be as in Theorem 13. Assume the density of $\left(\sqrt{r_{\alpha}}\ell^{\alpha}\left(\bm{\theta};\bm{\rho}\right)\right)_
{\bm{\rho}\in\mathcal{R},\alpha}$ is equicontinuous and bounded as all $\epsilon^{-1},t,N_{\alpha}\to\infty$. Then the joint density of $\left(\sqrt{r_{\alpha}}\ell^{\alpha}\left(\bm{\theta};\bm{\rho}\right)\right)_
{\bm{\rho}\in\mathcal{R},\alpha}$ is pointwise equal to that of $\left(\sqrt{r_{\alpha}}\hat{\ell}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)
\right)_{\bm{\rho}\in\mathcal{R},\alpha}$ up to an additive error $\operatorname{O}\left(\pi\right)$.

As-written our result only gives the loss function distribution at a single point $\bm{\theta}$ in parameter space. However, any finite set $\varTheta$ of points in parameter space can be considered by taking

$$ $\bm{\rho}\left(\bm{\theta}\right)=\bm{U}^{\alpha}\left(\bm{\theta}\right)\bm{ \rho}\bm{U}^{\alpha\dagger}\left(\bm{\theta}\right)$ (81) $$

to be elements in an augmented set of input states:

$$ $\mathcal{R}^{\prime}=\mathcal{R}\times\varTheta.$ (82) $$

We can give a more concrete form for the parameter-dependence of the loss landscape by considering the joint distribution of $\ell$ with its derivatives. We begin with the gradient. In order to consider the joint distribution over what can potentially be many gradient components—a number growing with the number of parameters $p$—we take the following additional assumption on the growth of $t$ with $p$ so we are able to fully capture correlations between the derivatives.

###### Assumption 15 (Scaling of parameter space dimension with $t$ -design) .

The number of trained parameters $p$ satisfies(^3^33We here use a bound on $p^{2}$ so that later we can reuse this assumption for the Hessian; at this stage we really only need this bound for $p$.):

$$ $p^{2}\leq\operatorname{o}\left(\min\left(\frac{\log\left(\epsilon^{-1}\right)} {\log\log\left(\epsilon^{-1}\right)},\frac{\sqrt{t}}{\log\left(t\right)}\right )\right),$ (83) $$

where the $\mathcal{G}_{i}^{\alpha}$ are i.i.d. $\epsilon$-approximate $t$-designs over $\mathcal{T}_{\alpha}=\operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right)$, and

$$ $p\leq\operatorname{O}\left(\operatorname{poly}\left(\min_{\alpha}N_{\alpha} \right)\right).$ (84) $$

It is possible to weaken this assumption and instead assume that the $\mathcal{G}_{i}^{\alpha}$ are $\epsilon$-approximate $2$-designs rather than $t$-designs, but this is at the expense of requiring $\mathcal{N}\leq\operatorname{o}\left(N^{\frac{2}{3}}\right)$ and only considering jointly at most $\sim\log\left(N\right)$ components of the gradient. We leave further details of this alternative setting to Appendix I.

It will also be convenient for the rest of our discussion to consider a concrete set of $\bm{A}_{i}$.

###### Assumption 16 (Concrete choice of $\bm{A}_{i}$ ) .

For $\mathbb{F}_{\alpha}=\mathbb{C},\mathbb{H}$, the $\bm{A}_{i}^{\alpha}$ are rank-$1$. For $\mathbb{F}_{\alpha}=\mathbb{R}$, the $\bm{A}_{i}^{\alpha}$ are rank-$2$.

The distinction between the cases $\mathbb{F}_{\alpha}=\mathbb{C},\mathbb{H}$ and $\mathbb{F}_{\alpha}=\mathbb{R}$ is due to there being no rank $1$ operators in the defining representation of $\mathfrak{so}\left(N_{\alpha}\right)$. This choice of $\bm{A}_{i}^{\alpha}$ may seem unphysical due to their nonlocality, but one can emulate the behavior of high-rank (e.g., Pauli) rotations by considering a factor of $N_{\alpha}$ more layers $p_{\alpha}$ in a given simple sector—each, for instance, performing a rotation under each eigenvector of $\bm{A}_{i}^{\alpha}$—and then tying the associated parameters together. This breaks no other assumptions as taking $p_{\alpha}\to N_{\alpha}p_{\alpha}$ maintains Eq. (84).

We then have the following theorem. It is stated assuming only a single input $\bm{\rho}$ which is rank-$1$ when projected to the defining representation of each simple sector to simplify the final result. However, in Appendix D.3 we give a full, exact expression of the joint distribution in terms of Wishart matrix elements for any choice of $\mathcal{R}\ni\bm{\rho}$ and independent of the scaling of $N_{\alpha}-r_{\alpha}$.

###### Theorem 17 (Gradient distribution) .

Consider the setting of Theorem 13, with the additional Assumptions 15 and 16. Assume $\frac{\sigma_{o}^{\alpha}}{\overline{o}}$ is bounded—where $\sigma_{o}^{\alpha}$ is the standard deviation of the eigenvalues of $\bm{O}^{\alpha}$—and assume $\left\lvert\mathcal{R}\right\rvert=1$ with element $\bm{\rho}$ such that each $\bm{\rho}^{\alpha}$ is rank-$1$ in its defining representation. We have the convergence in joint distributions:

$$ $\left(\mathcal{N}\ell\left(\bm{\theta};\bm{\rho}\right),\mathcal{N}\partial_{i }\ell\left(\bm{\theta};\bm{\rho}\right)\right)_{i\in\left[p\right]} \rightsquigarrow\left(\mathcal{N}\hat{\ell}\left(\bm{\theta};\bm{\rho}\right), \mathcal{N}\hat{\ell}_{;i}\left(\bm{\theta};\bm{\rho}\right)\right)_{i\in\left [p\right]}$ (85) $$

as all $\epsilon^{-1},t,N_{\alpha}\to\infty$, where conditioned on

$$ $\hat{\ell}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)=z^{\alpha}$ (86) $$

we have that

$$ $\hat{\ell}_{;i\mid z^{\alpha}}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)= \frac{2I_{\alpha}\sigma_{o}^{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right )}{N_{\alpha}}\sqrt{\frac{\beta_{\alpha}z^{\alpha}}{I_{\alpha}\overline{o}^{ \alpha}}}\chi_{i}^{\alpha}G_{i}^{\alpha},$ (87) $$

where

$$ $\hat{\ell}_{;i}\left(\bm{\theta};\bm{\rho}\right)\mid\left\{\hat{\ell}^{\alpha }\left(\bm{\theta},\bm{\rho}\right)=z^{\alpha}\right\}_{\alpha}\equiv\hat{\ell }_{;i\mid\bm{z}}\left(\bm{\theta};\bm{\rho}\right)\equiv\sum_{\alpha}\hat{\ell }_{;i\mid z^{\alpha}}^{\alpha}\left(\bm{\theta};\bm{\rho}\right).$ (88) $$

Here, the $\chi_{i}^{\alpha}$ are independent $\chi$-distributed random variables with $\max\left(2,\beta_{\alpha}\right)$ degrees of freedom and the $G_{i}^{\alpha}$ are i.i.d. standard normal random variables. In particular, the distributions differ in Lévy–Prokhorov metric by

$$ $\pi^{\prime}=\operatorname{O}\left(\frac{p\log\log\left(\mathcal{N}^{-t} \epsilon^{-1}\right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+\frac{p \log\left(t\right)}{\sqrt{t}}+\sum_{\alpha}\sqrt{\frac{\mathcal{N}\log\left(N_ {\alpha}\right)}{N_{\alpha}^{1.001}}}+\frac{\mathcal{N}^{\frac{1}{3}}\log\left (N\right)}{N^{\frac{5}{12}}}\right)=\operatorname{o}\left(1\right).$ (89) $$

This result is stronger than typical barren plateau results as it gives the full distributional form of the gradient—even when conditioned on the loss contribution $z^{\alpha}$—rather than just its variance [, , , ]. The conditional distribution as in Eq. (87) is illustrated in Fig. 2(b).

Just as with the loss function, we can strengthen Theorem 17 to show pointwise convergence in probability densities assuming equicontinuity of the original distribution.

###### Corollary 18 (Convergence of gradient densities) .

Let $\mathcal{J},\ell,\hat{\ell}$ be as in Corollary 14. Assume the density of $\left(\sqrt{r_{\alpha}}\ell_{;i}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)
\right)_{\alpha}$ is equicontinuous and bounded for all $z^{\alpha}\in\mathbb{R}_{\geq 0}$ as all $\epsilon^{-1},t,N_{\alpha}\to\infty$. Then the joint density of $\left(\sqrt{r_{\alpha}}\ell_{;i}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)
\right)_{\alpha}$ is pointwise equal to that of $\left(\sqrt{r_{\alpha}}\hat{\ell}_{;i}^{\alpha}\left(\bm{\theta};\bm{\rho}
\right)\right)_{\alpha}$ up to an additive error $\operatorname{O}\left(\pi^{\prime}\right)$.

We now give our final result, which specifies the joint distribution of not only the loss and gradient but also the Hessian at critical points. This is required to reason about the critical point distribution of the loss landscape using the so-called Kac–Rice formula [] that we will discuss in detail in Appendix C.3. We state our result assuming $\sigma_{o}\ll\overline{o}$ for simplicity, giving as we did for the gradient the full expression in terms of Wishart matrix elements in Appendix D.3.

###### Theorem 19 (Hessian distribution) .

Let $\mathcal{J},\ell,\hat{\ell}$ be as in Theorem 17. Assume all $\frac{\sigma_{o}^{\alpha}}{\overline{o}^{\alpha}}=\operatorname{o}\left(1\right)$ as $N_{\alpha}\to\infty$. We have the convergence in joint distributions:

$$ $\left(\ell\left(\bm{\theta};\bm{\rho}\right),\partial_{i}\ell\left(\bm{\theta} ;\bm{\rho}\right),\partial_{i}\partial_{j}\ell\left(\bm{\theta};\bm{\rho} \right)\right)_{\bm{\rho}\in\mathcal{R},i,j\in\left[p\right]}\rightsquigarrow \left(\hat{\ell}\left(\bm{\theta};\bm{\rho}\right),\hat{\ell}_{;i}\left(\bm{ \theta};\bm{\rho}\right),\hat{\ell}_{;i,j}\left(\bm{\theta};\bm{\rho}\right) \right)_{\bm{\rho}\in\mathcal{R},i,j\in\left[p\right]}$ (90) $$

as all $\epsilon^{-1},t,N_{\alpha}\to\infty$, where conditioned on

$$ $\hat{\ell}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)=z^{\alpha}$ (91) $$

and

$$ $\hat{\ell}_{;i\mid z^{\alpha}}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)=0$ (92) $$

we have that

$$ $\hat{\ell}_{;i,j\mid z^{\alpha},0}^{\alpha}\left(\bm{\theta};\bm{\rho}\right)= \frac{2I_{\alpha}\sigma_{o}^{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right )}{N_{\alpha}^{2}}\sqrt{\frac{z^{\alpha}}{I_{\alpha}\overline{o}_{\alpha}}}G_{ i}^{\alpha}\chi_{j}^{\alpha}\bra{i}\bm{W}^{\alpha}\ket{j},$ (93) $$

where

$$ $\hat{\ell}_{;i,j}\left(\bm{\theta};\bm{\rho}\right)\mid\left\{\hat{\ell}^{ \alpha}\left(\bm{\theta},\bm{\rho}\right)=z^{\alpha},\left(\hat{\ell}_{;i}^{ \alpha}\left(\bm{\theta};\bm{\rho}\right)\right)_{i\in\left[p\right]}=\bm{0} \right\}_{\alpha}\equiv\hat{\ell}_{;i,j\mid\bm{z},\bm{0}}\left(\bm{\theta};\bm {\rho}\right)\equiv\sum_{\alpha}\hat{\ell}_{;i,j\mid z^{\alpha},0}^{\alpha} \left(\bm{\theta};\bm{\rho}\right).$ (94) $$

Here, the $G_{i}^{\alpha}$ are i.i.d. standard normal random variables, the $\chi_{i}^{\alpha}$ are independent $\chi$-distributed random variables with $\max\left(2,\beta_{\alpha}\right)$ degrees of freedom, and the $\bm{W}^{\alpha}$ are independent Wishart-distributed random matrices with $\beta_{\alpha}r_{\alpha}$ degrees of freedom. In particular, the distributions differ in Lévy–Prokhorov metric by

$$ $\pi^{\prime\prime}=\operatorname{O}\left(\frac{p^{2}\log\log\left(\mathcal{N}^ {-t}\epsilon^{-1}\right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+ \frac{p^{2}\log\left(t\right)}{\sqrt{t}}+\sum_{\alpha}\frac{\mathcal{N}^{\frac {1}{3}}\log\left(N_{\alpha}\right)}{N_{\alpha}^{\frac{1}{3}}}\right)= \operatorname{o}\left(1\right).$ (95) $$

###### Definition 11 (Jordan algebraic Wishart system) .

###### Assumption 12 ( ϵ italic-ϵ \epsilon italic_ϵ -approximate t 𝑡 t italic_t -design of observable basis) .

###### Theorem 13 (Loss function distribution) .

###### Corollary 14 (Convergence of loss function densities) .

###### Assumption 15 (Scaling of parameter space dimension with t 𝑡 t italic_t -design) .

###### Assumption 16 (Concrete choice of 𝑨 i subscript 𝑨 𝑖 \bm{A}_{i} bold_italic_A start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) .

###### Theorem 17 (Gradient distribution) .

###### Corollary 18 (Convergence of gradient densities) .

###### Theorem 19 (Hessian distribution) .

## Appendix C Formal Discussion of the Consequences of Our Results

Before proceeding with proofs of our results we examine their implications.

### C.1 Barren Plateaus

The first implication of our results that we will discuss is the unification of barren plateau results [, ] in the large $N_{\alpha}$ limit.

###### Corollary 20 (General expression for the loss function variance) .

Let $\ell$ be as in Theorem 13. The variance is:

$$ $\operatorname{Var}\left[\ell\left(\bm{\theta};\bm{\rho}\right)\right]=\sum_{ \alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(\left( \bm{\rho}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(\mathfrak{g}_{ \alpha}\right)}+\operatorname{O}\left(\pi\right).$ (96) $$

Here, $\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)$ is the dimension of the automorphism group of $\mathcal{A}_{\alpha}$ and $\pi$ is as in Eq. (80)(^4^44Through careful examination of our proof this error is actually identically zero when $t\geq 2$ and $\epsilon=0$ in Assumption 12 as here we are only taking a second moment.).

###### Proof.

Note that

$$ $\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)=\left(\beta_{\alpha}-1 \right)N_{\alpha}+\frac{\beta_{\alpha}N_{\alpha}\left(N_{\alpha}-1\right)}{2}= \frac{\beta_{\alpha}N_{\alpha}^{2}}{2}+\operatorname{O}\left(N_{\alpha}\right).$ (97) $$

Similarly,

$$ $\frac{\left(\overline{o}^{\alpha}\right)^{2}}{r_{\alpha}}=\frac{\left\lVert\bm {O}^{\alpha}\right\rVert_{\text{F}}^{2}}{N_{\alpha}^{2}}.$ (98) $$

By Eq. (46), we also have that

$$ $I_{\alpha}\left\lVert\bm{O}^{\alpha}\right\rVert_{\text{F}}^{2}=\Tr\left(\left (\bm{O}^{\alpha}\right)^{2}\right),$ (99) $$

with $\Tr\left(\cdot\right)$ the trace on the full $N$-dimensional Hilbert space. In the language of Ragone *et al.* [] this is the $\mathcal{A}_{\alpha}$-purity $\mathcal{P}_{\mathcal{A}_{\alpha}}$ of $\bm{O}$. Using the fact that the diagonal entries of $\beta_{\alpha}\bm{W}^{\alpha}$ are i.i.d. $\chi^{2}$-distributed with $\beta_{\alpha}r_{\alpha}$ degrees of freedom (see Appendix A.4), we then have from Theorem 13 that as $N_{\alpha}\to\infty$:

$$ $\displaystyle\operatorname{Var}\left[\ell\left(\bm{\theta};\bm{\rho}\right)\right]$ $\displaystyle=\sum_{\alpha}\operatorname{Var}\left[\hat{\ell}^{\alpha}\left( \bm{\theta};\bm{\rho}\right)\right]+\operatorname{O}\left(\pi\right)$ (100) $\displaystyle=\sum_{\alpha}\frac{2I_{\alpha}^{2}\left(\overline{o}^{\alpha} \right)^{2}}{\beta_{\alpha}r_{\alpha}}\Tr_{\alpha}\left(\left(\bm{\rho}^{ \alpha}\right)^{2}\right)+\operatorname{O}\left(\pi\right)$ $\displaystyle=\sum_{\alpha}\frac{2I_{\alpha}\left(\overline{o}^{\alpha}\right) ^{2}}{\beta_{\alpha}r_{\alpha}}\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2} \right)+\operatorname{O}\left(\pi\right)$ $\displaystyle=\sum_{\alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2} \right)\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R}} \left(\mathfrak{g}_{\alpha}\right)}+\operatorname{O}\left(\pi\right).$ $$

∎

This result immediately implies Theorem 1 of Ragone *et al.* [] in the case of Lie algebra-supported ansatzes. However, our result holds for *all* variational ansatzes, i.e., it does not rely on either $\mathrm{i}\bm{O}$ or $\mathrm{i}\bm{\rho}$ being a member of the dynamical Lie algebra generating the ansatz.

###### Corollary 20 (General expression for the loss function variance) .

###### Proof.

### C.2 The Quantum Neural Tangent Kernel

We now connect our results to the quantum neural tangent kernel (QNTK) literature [, , , , , ]. The landmark result in this field is that, under certain conditions on $\mathcal{R}$, $\bm{O}$, and $\mathcal{A}$, variational loss functions are asymptotically a Gaussian process when $\epsilon^{-1},t,N\to\infty$. However, this same body of work has noted that such a Gaussian process description cannot generally be true: for instance, if $\bm{O}$ is rank-$1$ and $\mathcal{A}$ is the space of complex Hermitian matrices, the loss should be Porter–Thomas distributed as this reduces to a random circuit sampling setting [].

Our results can be seen as a unifying model of neural network loss landscapes, including both when convergence to a Gaussian process is achieved and when it is not. Recall that Theorem 13 demonstrated that the asymptotic expression for the variational loss with objective observable $\bm{O}$ is distributed as a Wishart process:

$$ $\hat{\ell}\left(\bm{\rho}\right)=\sum_{\alpha}\frac{I_{\alpha}\overline{o}^{ \alpha}}{r_{\alpha}}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\bm{W}^{\alpha}\right),$ (101) $$

even when rescaled by a quantity $\mathcal{N}$ exponentially large in the problem size. This correctly captures the Porter–Thomas behavior when $\mathcal{A}$ is the space of complex Hermitian matrices and $\rank\left(\bm{O}\right)=r=1$. This is because the diagonal entries of a complex Wishart matrix are $\chi^{2}$-distributed with two degrees of freedom each, which is identical to the exponential distribution(^5^55It is also easy to check when $r=1$ and the ansatz unitaries are Haar random that the assumptions we use in proving our results hold for any $\mathcal{N}\leq\operatorname{O}\left(N\right)$.).

Indeed, our more general result can be used to *exactly* characterize when QNNs asymptotically form Gaussian processes. This occurs when the loss is scaled by some $\mathcal{N}$ such that Eq. (101) has nonvanishing variance yet the higher-order cumulants vanish. We compute the third-order cumulant:

$$ $\kappa_{3}\left(\mathcal{N}\hat{\ell}\left(\bm{\rho}\right)\right)= \operatorname{O}\left(\mathcal{N}^{3}\sum\limits_{\alpha}\frac{I_{\alpha}^{3} \left(\overline{o}^{\alpha}\right)^{3}}{r_{\alpha}^{2}}\Tr_{\alpha}\left(\left (\bm{\rho}^{\alpha}\right)^{3}\right)\right).$ (102) $$

We thus can state this pair of conditions formally as follows.

###### Corollary 21 (Exact conditions for convergence to a Gaussian process) .

Let $\mathcal{J},\ell$ be as in Theorem 13. $\mathcal{N}\ell\left(\bm{\theta};\bm{\rho}\right)$ is asymptotically a Gaussian process as $\epsilon^{-1},t,N\to\infty$ if and only if

$$ $\mathcal{N}\leq\operatorname{o}\left(\min\limits_{\alpha}\frac{r_{\alpha}^{ \frac{2}{3}}}{I_{\alpha}\overline{o}^{\alpha}\Tr_{\alpha}\left(\left(\bm{\rho} ^{\alpha}\right)^{3}\right)^{\frac{1}{3}}}\right)$ (103) $$

and $\mathcal{N}^{2}\operatorname{Var}\left[\ell\left(\bm{\theta};\bm{\rho}\right)\right]$ (with the variance as given in Corollary 20) is nonvanishing for all $\bm{\rho}\in\mathcal{R}$.

When these conditions are met, the covariance is given by:

$$ $\displaystyle\mathcal{K}\left(\bm{\rho},\bm{\rho^{\prime}}\right)$ $\displaystyle=\lim_{N\to\infty}\mathcal{K}_{N}\left(\bm{\rho},\bm{\rho^{\prime }}\right)$ (104) $\displaystyle=\lim_{N\to\infty}\mathcal{N}^{2}\sum_{\alpha}\frac{\beta_{\alpha }\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}{2\dim_{\mathbb{R}}\left( \mathfrak{g}_{\alpha}\right)r_{\alpha}}I_{\alpha}\mathbb{E}\left[\Tr_{\alpha} \left(\bm{\rho}^{\alpha}\left(\bm{W}^{\alpha}-r_{\alpha}\right)\right)\Tr_{ \alpha}\left(\bm{\rho^{\prime}}^{\alpha}\left(\bm{W}^{\alpha}-r_{\alpha}\right )\right)\right].$ $$

It is now easy to see why (for simple $\mathcal{A}$) the $r=1$ case does not converge to a Gaussian process: when $r=1$ we may only choose a normalization $\mathcal{N}\leq\operatorname{o}\left(N\right)$ such that higher-order cumulants vanish asymptotically, but then the variance vanishes asymptotically. In contrast, our results demonstrate convergence to a Wishart process through *any* finite number of cumulants, not just the first two.

We can also be more concrete on the form of $\mathcal{K}\left(\bm{\rho},\bm{\rho^{\prime}}\right)$. Indeed, we are able to show that each algebraic sector $\mathcal{A}_{\alpha}$ contributes to the covariance an explicit, quadratic expression in $\bm{\rho}^{\alpha}$ and $\bm{\rho^{\prime}}^{\alpha}$, depending only on the field $\mathbb{F}_{\alpha}$ associated with its defining representation. In other words, whether or not a given quantum neural network is asymptotically a Gaussian process just depends on how SWAP tests of the inputs scale. As the expression is complicated we do not reproduce it here, instead giving references to where it may be found for each of $\mathbb{F}_{\alpha}=\mathbb{R},\mathbb{C},\mathbb{H}$.

###### Theorem 22 ( $\mathcal{K}$ is quadratic in $\bm{\rho},\bm{\rho^{\prime}}$ ) .

$\mathcal{K}\left(\bm{\rho},\bm{\rho^{\prime}}\right)$ can be written as a closed-form, explicit function of only the $\Tr\left(\bm{\rho}^{\alpha}\right)$, $\Tr\left(\bm{\rho^{\prime}}^{\alpha}\right)$, $\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)$, $\Tr\left(\left(\bm{\rho^{\prime}}^{\alpha}\right)^{2}\right)$, and $\Tr\left(\bm{\rho}^{\alpha}\bm{\rho^{\prime}}^{\alpha}\right)$.

###### Proof.

By the Haar invariance of $\bm{W}^{\alpha}$ there are explicit formulas for covariances of the form of $\mathcal{K}\left(\bm{\rho},\bm{\rho^{\prime}}\right)$ as degree-$2$ polynomials in $\bm{\rho}$ and $\bm{\rho^{\prime}}$. These formulas are given in:

- 1.
Theorem 1 of Redelmeier [] when $\mathbb{F}=\mathbb{R}$;
- 2.
Theorem 3.5 of Mingo and Speicher [] when $\mathbb{F}=\mathbb{C}$;
- 3.
Sec. 5.3 of Redelmeier [] when $\mathbb{F}=\mathbb{H}$.

The result then follows by noting that $I_{\alpha}\Tr_{\alpha}\left(\cdot^{\alpha}\right)=\Tr\left(\cdot^{\alpha}\right)$.
∎

We give a simple example of the calculation of $\mathcal{K}$ when all inputs $\bm{\rho_{0}}$ can be mutually diagonalized. By the unitary invariance of Wishart matrices we can assume WLOG that all inputs are diagonal. Inputs are then completely parameterized by the eigenvalues $\bm{x}\in\mathbb{R}_{\geq 0}^{N_{\alpha}}$ of the input states, where $\left\lVert\bm{x}\right\rVert_{1}=1$. By the independence of diagonal entries of a Wishart matrix we then have that (when $\mathcal{N}=1$):

$$ $\displaystyle\mathcal{K}_{N}\left(\bm{x},\bm{x^{\prime}}\right)$ $\displaystyle=\mathcal{N}^{2}\sum\limits_{\alpha}\frac{\beta_{\alpha}\Tr\left( \left(\bm{O}^{\alpha}\right)^{2}\right)}{2\dim_{\mathbb{R}}\left(\mathfrak{g}_ {\alpha}\right)r_{\alpha}}\sum_{i=1}^{N}x_{i}^{\alpha}x_{i}^{\prime\alpha} \operatorname{Var}\left[\bra{i}\bm{W}^{\alpha}\ket{i}\right]+\operatorname{o} \left(1\right)$ (105) $\displaystyle=\mathcal{N}^{2}\sum\limits_{\alpha}\frac{\Tr\left(\left(\bm{O}^{ \alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right) }\bm{x}^{\alpha}\cdot\bm{x}^{\prime\alpha}+\operatorname{o}\left(1\right),$ $$

where in the final line we used similar simplifications as in Appendix C.1. Noting that $\bm{x}^{\alpha}\cdot\bm{x}^{\prime\alpha}$ is just the overlap of $\bm{\rho}^{\alpha}$ and $\bm{\rho^{\prime}}^{\alpha}$ yields an equivalent formulation:

$$ $\mathcal{K}_{N}\left(\bm{\rho},\bm{\rho^{\prime}}\right)=\mathcal{N}^{2}\sum \limits_{\alpha}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)}{\dim_{ \mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)}\Tr\left(\bm{\rho}^{\alpha}\bm{ \rho^{\prime}}^{\alpha}\right)+\operatorname{o}\left(1\right)=\sum\limits_{ \alpha}\operatorname{O}\left(\frac{\mathcal{N}^{2}r_{\alpha}}{N_{\alpha}^{2}} \right)\Tr\left(\bm{\rho}^{\alpha}\bm{\rho^{\prime}}^{\alpha}\right)+ \operatorname{o}\left(1\right).$ (106) $$

In particular, when $\mathcal{A}$ is simple, a $\mathcal{N}=\operatorname{\Omega}\left(\frac{N}{\sqrt{r}}\right)$ normalization is required for the network to asymptotically form a Gaussian process over pure inputs.

While previous results have shown that Gaussian processes efficiently train, we argue that this may paint an overly optimistic picture for generic QNNs. Focusing on the case where the ansatz is an $\operatorname{\Theta}\left(\log\left(n\right)\right)$-depth, $2$-dimensional circuit on $n$ qubits, we have that the QNN forms a Gaussian process when:

$$ $\mathcal{N}=\exp\left(\operatorname{\Omega}\left(\log\left(n\right)^{2}\right) \right),$ (107) $$

i.e., at a normalization superpolynomial in $n$. Thus while it is known that such networks can achieve a constant improvement in $\mathcal{N}\hat{\ell}$ in time polynomial in $n$ via gradient descent [], this translates to a superpolynomially vanishing improvement in the loss function value when in the physical normalization of $\mathcal{N}=1$. More generally, the covariance we here derive can be used with results in the classical neural tangent kernel literature to reason about the training behavior [, , , ] and generalization ability [, ] of this class of quantum neural networks, which we hope to analyze in more detail in the future.

###### Corollary 21 (Exact conditions for convergence to a Gaussian process) .

###### Theorem 22 ( 𝒦 𝒦 \mathcal{K} caligraphic_K is quadratic in 𝝆 , 𝝆 ′ 𝝆 superscript 𝝆 bold-′ \bm{\rho},\bm{\rho^{\prime}} bold_italic_ρ , bold_italic_ρ start_POSTSUPERSCRIPT bold_′ end_POSTSUPERSCRIPT ) .

###### Proof.

### C.3 Local Minima

We end by examining the distribution of local minima of the loss landscape. This has been done previously in the more restricted setting where no structure is imposed on the loss function [, ]. In this section we assume the assumptions of Theorem 19 and that as all $\epsilon^{-1},t,N_{\alpha}\to\infty$,

$$ $\gamma_{\alpha}\equiv\frac{p_{\alpha}}{\beta_{\alpha}r_{\alpha}}$ (108) $$

is held constant. Here, $p_{\alpha}$ is the number of $\bm{A}_{i}$ that are nontrivial on the simple component $\mathcal{A}_{\alpha}$ of $\mathcal{A}$. $\gamma_{\alpha}$ is the so-called *overparameterization ratio* discussed in Anschuetz [], Anschuetz and Kiani []. We also assume for the simplicity of our expressions that each simple sector is fully controllable, i.e., that each ansatz generator $\bm{A}_{i}$ is only nontrivial on a single simple component of $\mathcal{A}$. In principle a more general expression could be written as well, though the gradient density would be a convolution over complicated probability densities, and the Hessian distributed as a free convolution of multiple Wishart matrices.

We calculate the distribution of local minima via the *Kac–Rice formula* [], which gives the expected density of local minima of a function $\ell$ at a function value $z$. Anschuetz [] demonstrated that the assumptions for the Kac–Rice formula are satisfied for variational loss landscapes, and when rotationally invariant on the $p$-torus takes the form:

$$ $\mathbb{E}\left[\operatorname{Crt}_{0}\left(z\right)\right]=\left(2\uppi\right )^{p}\mathbb{E}\left[\det\left(\bm{H}_{z}\right)\bm{1}\left\{\bm{H}_{z}\succeq \bm{0}\right\}\right]\mathbb{P}\left[\bm{G}_{z}=\bm{0}\right]\mathbb{P}\left[ \ell=z\right].$ (109) $$

Here, $\mathbb{E}\left[\operatorname{Crt}_{0}\left(z\right)\right]$ is the expected density of local minima at a function value $z$, $\bm{G}_{z}$ is the gradient conditioned on $\ell=z$, and $\bm{H}_{z}$ is the Hessian conditioned on $\ell=z$ and $\bm{G}=\bm{0}$. In a slight abuse of notation, here $\mathbb{P}\left[\cdot\right]$ denotes the probability density associated with the event $\cdot$.

We can evaluate this expression using Theorems 13, 17, and 19.

###### Corollary 23 (Density of local minima to multiplicative leading order) .

Let $\mathcal{J},\hat{\ell}$ be as in Corollary 18 and Theorem 19. Assume each ansatz generator $\bm{A}_{i}$ is only nontrivial on a single simple component of $\mathcal{A}$, and further assume that $\bm{\rho}^{\alpha}$ is rank-$1$ in its defining representation. Assume as well that the overparameterization ratios:

$$ $\gamma_{\alpha}\equiv\frac{p_{\alpha}}{\beta_{\alpha}r_{\alpha}}$ (110) $$

remain fixed as $N\to\infty$. Let $\mu_{\bm{H}_{z}^{\alpha}}$ be the empirical spectral measure of

$$ $\bm{\tilde{H}}_{z}^{\alpha}\equiv N_{\alpha}^{-1}\bm{S}^{\alpha}\odot\bm{W}^{ \alpha},$ (111) $$

where (for $i\geq j$) $S_{i,j}^{\alpha}=\left\lvert G_{i}^{\alpha}\right\rvert\chi_{j}^{\alpha}$ with random variables defined as in Theorem 19 and $\odot$ denotes the Hadamard product. Let $\lambda_{z,\alpha}^{\ast}$ denote the infinimum of the support of $\mu_{\bm{H}_{z}^{\alpha}}$. Then the expected density of local minima of $\hat{\ell}$ at a loss function value $z$ is:

$$ $\mathbb{E}\left[\operatorname{Crt}\left(z\right)\right]=\operatorname*{ \scalerel*{\ast}{\sum}}_{\alpha}\mathbb{E}\left[\operatorname{Crt}_{0}^{\alpha }\right]\left(z\right),$ (112) $$

where

$$ $\displaystyle p^{-1}\ln\left(\mathbb{E}\left[\operatorname{Crt}_{0}^{\alpha} \left(z\right)\right]\right)=$ $\displaystyle\ln\left(\frac{\uppi\max\left(2,\beta_{\alpha}\right)}{4\sqrt{ \beta_{\alpha}}}\right)+\frac{1}{2\gamma_{\alpha}}\left(1-\frac{z}{\Tr\left( \bm{\rho}^{\alpha}\right)\overline{o}^{\alpha}}+\ln\left(\frac{z}{\Tr\left(\bm {\rho}^{\alpha}\right)\overline{o}^{\alpha}}\right)\right)$ (113) $\displaystyle+p^{-1}\ln\mathbb{E}\left[\exp\left(p\int\differential{\mu_{\bm{ \tilde{H}}_{z}^{\alpha}}\left(\lambda\right)}\ln\left(\lambda\right)\right)\bm {1}\left\{\lambda_{z,\alpha}^{\ast}\geq 0\right\}\right]+\operatorname{o}\left (1\right).$ $$

###### Proof.

We will first consider a single simple component $\mathcal{A}_{\alpha}$ with pure $\bm{\rho}^{\alpha}$ (and with $\alpha$ labels implicit for clarity of notation), and describe in the end how a full distribution of local minima can be determined from this via a convolution. From Corollary 14 it follows that the density of the loss is(^6^66Technically we need to consider the density of the loss rescaled by $\sqrt{N}$ to achieve pointwise convergence in densities, but the results are equivalent after rescaling $z$.):

$$ $\mathbb{P}\left[\hat{\ell}=z\right]=\frac{\left(\frac{\beta rz}{2I\overline{o} }\right)^{\frac{\beta r}{2}-1}\exp\left(-\frac{\beta rz}{2I\overline{o}}\right )}{2\operatorname{\Gamma}\left(\frac{\beta r}{2}\right)}.$ (114) $$

Similarly, using Corollary 18 and recalling that $\beta$ is one of $1$, $2$, or $4$, we can evaluate the density of the gradient at zero:

$$ $\mathbb{P}\left[\bm{G}_{z}=\bm{0}\right]=\left(\frac{1}{\sqrt{2\uppi}\sigma_{z }}\int_{0}^{\infty}\differential{x}\frac{f_{\chi^{\max\left(2,\beta\right)}} \left(x\right)}{x}\right)^{p}=\left(\frac{\max\left(2,\beta\right)}{4\sigma_{z }}\right)^{p},$ (115) $$

where $f_{\chi^{\max\left(2,\beta\right)}}$ is the density of a $\chi$-distributed random variable with $\max\left(2,\beta\right)$ degrees of freedom and

$$ $\sigma_{z}=\frac{2I\sigma_{o}}{N}\sqrt{\frac{\beta z}{I\overline{o}}}.$ (116) $$

Finally, we consider the Hessian determinant. Recall Theorem 19 for the Hessian components, which gives the Hessian as:

$$ $\frac{N}{2I\sigma_{o}}\sqrt{\frac{I\overline{o}}{z}}\bm{H}_{z}=N^{-1}\bm{ \tilde{S}}\odot\bm{W},$ (117) $$

where $\tilde{S}_{i,j}=G_{i}\chi_{j}$. We now claim that $\bm{H}_{z}$ can only be positive semidefinite if all $G_{i}\geq 0$. To see this, note that $\chi_{i}W_{i,i}$ is always nonnegative. When $G_{i}<0$, then, the $\left(i,i\right)$ entry of $\bm{\tilde{S}}\odot\bm{W}$ is negative and thus is not positive semidefinite. We therefore can consider $\left\lvert G_{i}\right\rvert$ rather than $G_{i}$ up to pulling out a factor of $2^{-p}$ from the expectation. Putting everything together, to multiplicative leading order as $N\to\infty$,

$$ $\displaystyle p^{-1}\ln\left(\mathbb{E}\left[\operatorname{Crt}_{0}\left(z \right)\right]\right)=$ $\displaystyle\ln\left(\frac{\uppi\max\left(2,\beta\right)}{4\sqrt{\beta}} \right)+\frac{1}{2\gamma}\left(1-\frac{z}{I\overline{o}}+\ln\left(\frac{z}{I \overline{o}}\right)\right)$ (118) $\displaystyle+p^{-1}\ln\mathbb{E}\left[\exp\left(p\int\differential{\mu_{\bm{ \tilde{H}}_{z}}\left(\lambda\right)}\ln\left(\lambda\right)\right)\bm{1}\left \{\lambda_{z}^{\ast}\geq 0\right\}\right]+\operatorname{o}\left(1\right).$ $$

This completes our proof for a single simple component $\mathcal{A}_{\alpha}$ with pure $\bm{\rho}^{\alpha}$. To calculate the loss landscape of a JAWS associated with a nonsimple Jordan algebra, note that:

$$ $\mathbb{P}\left[\ell=z\right]=\operatorname*{\scalerel*{\ast}{\sum}}_{\alpha} \mathbb{P}\left[\ell^{\alpha}=z^{\alpha}\right],$ (119) $$

where $\operatorname*{\scalerel*{\ast}{\sum}}$ denotes convolution. The relative weights of the various sectors introduced by $\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right)$ can be accounted for by taking:

$$ $I_{\alpha}\to I_{\alpha}\Tr_{\alpha}\left(\bm{\rho}^{\alpha}\right)=\Tr\left( \bm{\rho}^{\alpha}\right).$ (120) $$

From Eq. (109) and our simplifying assumptions, then,

$$ $\mathbb{E}\left[\operatorname{Crt}_{0}\left(z\right)\right]=\operatorname*{ \scalerel*{\ast}{\sum}}_{\alpha}\mathbb{E}\left[\operatorname{Crt}_{0}^{\alpha }\right]_{I_{\alpha}\to\Tr\left(\bm{\rho}^{\alpha}\right)}\left(z\right),$ (121) $$

where $\mathbb{E}\left[\operatorname{Crt}_{0}^{\alpha}\right]$ denotes Eq. (118) associated with the algebraic sector $\mathcal{A}_{\alpha}$.
∎

While Corollary 23 is exact, it is obtuse almost to the point of obscurity, particularly due to the expectation over the Hessian. The obstruction to further simplification is the presence of the Hadamard product between $\bm{S}^{\alpha}$ and $\bm{W}$ in $\tilde{\bm{H}}_{z}^{\alpha}$, which is difficult to handle analytically. To get around this, we consider a slightly modified quantity where we condition both sides of Eq. (109) on the events:

$$ $\left\lvert G_{i}\right\rvert=\chi_{i}.$ (122) $$

In effect this can be considered as a regularization scheme, where new parameters $\tilde{\theta}_{i}$ are introduced as Lagrange multipliers with associated derivatives(^7^77The Kac–Rice formula as stated in Adler and Taylor [] allows one to consider this modified gradient jointly with the original loss.)

$$ $\tilde{\ell}_{;i}\equiv\tilde{\theta}_{i}+\left\lvert G_{i}\right\rvert-\chi_{i}$ (123) $$

and we consider a sufficiently small neighborhood of $\tilde{\theta}_{i}=0$. In this setting the nontrivial components of $\bm{\tilde{H}}_{z}^{\alpha}$ take the much more manageable form:

$$ $\bm{\tilde{H}}_{z}^{\alpha}=N_{\alpha}^{-1}\sqrt{\bm{\varSigma}^{\alpha}}\bm{W }^{\alpha}\sqrt{\bm{\varSigma}^{\alpha}},$ (124) $$

where $\bm{\varSigma}^{\alpha}$ is a diagonal matrix with entries i.i.d. $\chi^{2}$-distributed with $\max\left(2,\beta_{\alpha}\right)$ degrees of freedom. Analyzing the expected determinant of this random matrix leads us to prove the following.

###### Corollary 24 (Density of local minima to multiplicative leading order, regularized) .

Consider the setting of Corollary 23 conditioned on $\left\lvert G_{i}\right\rvert=\chi_{i}$ such that $\bm{\tilde{H}}_{z}^{\alpha}$ is as in Eq. (124). Let $\mu_{\text{MP}}^{\gamma}$ be the Marčenko–Pastur distribution with parameter $\gamma$. Then the expected density of local minima of $\hat{\ell}$ at a loss function value $z$ is:

$$ $\mathbb{E}\left[\operatorname{Crt}\left(z\right)\right]=\operatorname*{ \scalerel*{\ast}{\sum}}_{\alpha:\gamma_{\alpha}<1}\mathbb{E}\left[ \operatorname{Crt}_{0}^{\alpha}\right]\left(z\right),$ (125) $$

where:

$$ $\displaystyle p^{-1}\ln\left(\mathbb{E}\left[\operatorname{Crt}_{0}^{\alpha} \left(z\right)\right]\right)=$ $\displaystyle\ln\left(\frac{\uppi\max\left(2,\beta_{\alpha}\right)}{2\sqrt{ \beta_{\alpha}}}\right)+\frac{1}{2\gamma_{\alpha}}\left(1-\frac{z}{\Tr\left( \bm{\rho}^{\alpha}\right)\overline{o}_{\alpha}}+\ln\left(\frac{z}{\Tr\left(\bm {\rho}^{\alpha}\right)\overline{o}_{\alpha}}\right)\right)$ (126) $\displaystyle+\frac{\max\left(2,\beta_{\alpha}\right)}{2}-1-\upgamma+\int \differential{\mu_{\text{MP}}^{\gamma_{\alpha}}\left(\lambda\right)}\ln\left( \lambda\right)+\operatorname{o}\left(1\right).$ $$

###### Proof.

As in Corollary 23 we focus on a single simple component $\mathcal{A}_{\alpha}$. $\bm{\varSigma}$ is positive definite with probability $1$. As it is diagonal with i.i.d. $\chi^{2}$-distributed random variables each with $\max\left(2,\beta\right)$ degrees of freedom,

$$ $\exp\left(p\int\differential{\mu_{\bm{\varSigma}}}\left(\lambda\right)\ln\left (\lambda\right)\right)=\exp\left(p\left(\frac{\max\left(2,\beta\right)}{2}-1- \upgamma+\ln\left(2\right)\right)\right),$ (127) $$

where $\differential{\mu_{\bm{\varSigma}}}$ is the empirical spectral distribution of $\bm{\varSigma}$ and $\upgamma$ is the Euler–Mascheroni constant. What remains to be considered is the spectrum of

$$ $\bm{D}_{z}\equiv\bm{\varSigma}^{-\frac{1}{2}}\bm{\tilde{H}}_{z}\bm{\varSigma}^ {-\frac{1}{2}}=N^{-1}\bm{W}=\left(\frac{\beta r}{N}\right)\left(\beta r\right) ^{-1}\bm{W}.$ (128) $$

Recall that by assumption $\frac{\sigma_{o}}{\overline{o}}=\operatorname{o}\left(1\right)$ so by, e.g., Lemma 25 (proved in Appendix D.1):

$$ $\ln\left(\frac{\beta r}{N}\right)=\operatorname{o}\left(1\right).$ (129) $$

We need only focus on $\left(\beta r\right)^{-1}\bm{W}$, then. As discussed in Appendix A.4, this random matrix has empirical eigenvalue spectrum weakly converging almost surely to the Marčenko–Pastur distribution with parameter $\gamma$. In principle, large deviations in this convergence—even if they occur with exponentially small probability—can contribute corrections to the expected determinant due to its exponential sensitivity on the eigenvalues of $\bm{D}_{z}$. However, we show in Appendix K that these large deviations are dominated in the expectation by the Marčenko–Pastur distribution. Noting that the Marčenko–Pastur distribution with parameter $\gamma$ has support at the origin if and only if $\gamma>1$ and taking convolutions over simple algebraic sectors as in the proof of Corollary 23 then yields the final result.
∎

Dropping multiplicatively subleading factors from Eq. (126), we effectively have demonstrated that the density $\kappa_{\alpha}\left(z\right)$ of local minima for a given simple component $\mathcal{A}_{\alpha}$ is asymptotically given by:

$$ $\kappa_{\alpha}\left(z\right)=f_{\Gamma}\left(\frac{z}{\Tr\left(\bm{\rho}^{ \alpha}\right)\overline{o}^{\alpha}};\frac{p_{\alpha}}{2\gamma_{\alpha}},\frac {2\gamma_{\alpha}}{p_{\alpha}}\right)$ (130) $$

if $\gamma_{\alpha}<1$, and otherwise:

$$ $\kappa_{\alpha}\left(z\right)=\operatorname{\delta}\left(z\right).$ (131) $$

We here have used the expression for the density of the gamma distribution:

$$ $f_{\Gamma}\left(x;k,\theta\right)=\frac{1}{\operatorname{\Gamma}\left(k\right) \theta^{k}}x^{k-1}\exp\left(-\frac{x}{\theta}\right).$ (132) $$

Convolving $\kappa_{\alpha}$ over many simple sectors thus yields the final density for $z>0$:

$$ $\kappa\left(z\right)=\operatorname*{\scalerel*{\ast}{\sum}}_{\alpha:\gamma_{ \alpha}<1}f_{\Gamma}\left(\frac{z}{\overline{o}^{\alpha}\Tr\left(\bm{\rho}^{ \alpha}\right)};\frac{\beta_{\alpha}r_{\alpha}}{2},\frac{2}{\beta_{\alpha}r_{ \alpha}}\right).$ (133) $$

This distribution is illustrated in Fig. 3 for various parameter regimes. To multiplicative leading order in $\gamma$ this agrees exactly with the asymptotic local minima distribution studied in Anschuetz [], Anschuetz and Kiani []. See for instance Eq. (1) of Anschuetz [], which studies the case $\mathcal{A}=\mathcal{H}^{N}\left(\mathbb{C}\right)$; i.e., one takes $\beta_{\alpha}\to 2$, $r_{\alpha}\to m$, and $\frac{z}{\Tr\left(\bm{\rho}^{\alpha}\right)\overline{o}_{\alpha}}\to x$ to translate from our setting to their setting.

We can simplify this expression even further by noting that, asymptotically, the convolution of many gamma distributions is also gamma-distributed by the Welch–Satterthwaite equation [, ]. This yields:

$$ $\kappa\left(z\right)=f_{\Gamma}\left(z;k_{\text{eff}},\theta_{\text{eff}} \right),$ (134) $$

where

$$ $\displaystyle k_{\text{eff}}$ $\displaystyle=\frac{\left(\sum_{\alpha:\gamma_{\alpha}<1}\overline{o}^{\alpha} \Tr\left(\bm{\rho}^{\alpha}\right)\right)^{2}}{\sum_{\alpha:\gamma_{\alpha}<1} \frac{2\left(\overline{o}^{\alpha}\right)^{2}\Tr\left(\bm{\rho}^{\alpha}\right )^{2}}{\beta_{\alpha}r_{\alpha}}}\xrightarrow{\left\{N_{\alpha}\to\infty\right \}_{\alpha}}\overline{\ell}_{\text{u.p.}}^{2}\left(\sum_{\alpha:\gamma_{\alpha }<1}\frac{\Tr\left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(\left(\bm{ \rho}^{\alpha}\right)^{2}\right)}{\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha} \right)}\right)^{-1},$ (135) $\displaystyle\theta_{\text{eff}}$ $\displaystyle=\frac{\sum_{\alpha:\gamma_{\alpha}<1}\frac{2\left(\overline{o}^{ \alpha}\right)^{2}\Tr\left(\bm{\rho}^{\alpha}\right)^{2}}{\beta_{\alpha}r_{ \alpha}}}{\sum_{\alpha:\gamma_{\alpha}<1}\overline{o}^{\alpha}\Tr\left(\bm{ \rho}^{\alpha}\right)}\xrightarrow{\left\{N_{\alpha}\to\infty\right\}_{\alpha} }\overline{\ell}_{\text{u.p.}}^{-1}\sum_{\alpha:\gamma_{\alpha}<1}\frac{\Tr \left(\left(\bm{O}^{\alpha}\right)^{2}\right)\Tr\left(\left(\bm{\rho}^{\alpha} \right)^{2}\right)}{\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)}.$ (136) $$

Here,

$$ $\overline{\ell}_{\text{u.p.}}\equiv\sum_{\alpha:\gamma_{\alpha}<1}\overline{o} ^{\alpha}\Tr\left(\bm{\rho}^{\alpha}\right)$ (137) $$

is the mean loss function value over the underparameterized sectors, and the limit in each line is due to the identities (recalling that here we assume that $\bm{\rho}^{\alpha}$ is rank-$1$ in its defining representation):

$$ $\displaystyle\left(\overline{o}^{\alpha}\right)^{2}\Tr\left(\bm{\rho}^{\alpha} \right)^{2}=I_{\alpha}^{2}\left(\overline{o}^{\alpha}\right)^{2}\Tr_{\alpha} \left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)$ $\displaystyle=I_{\alpha}\left(\overline{o}^{\alpha}\right)^{2}\Tr\left(\left( \bm{\rho}^{\alpha}\right)^{2}\right),$ (138) $\displaystyle I_{\alpha}\left(\overline{o}^{\alpha}\right)^{2}=\frac{I_{\alpha }}{N_{\alpha}^{2}}\Tr_{\alpha}\left(\bm{O}^{\alpha}\right)^{2}=\frac{I_{\alpha }r_{\alpha}}{N_{\alpha}^{2}}\Tr_{\alpha}\left(\left(\bm{O}^{\alpha}\right)^{2}\right)$ $\displaystyle=\frac{r_{\alpha}}{N_{\alpha}^{2}}\Tr\left(\left(\bm{O}^{\alpha} \right)^{2}\right),$ (139) $$

as well as the identities considered in Appendix C.1.

Intriguingly, the relevant features of this density in the underparameterized regime are controlled by the $\mathcal{A}_{\alpha}$-purities of both $\bm{O}$ and $\bm{\rho}$ in the sectors in which they are underparameterized. This is the same quantity which controls the variance of the loss function (see Appendix C.1). However, even when there are no barren plateaus in the loss landscape—for instance, if the variance of the loss function are polynomially vanishing in $N$—the density $\kappa\left(z\right)$ may still have exponentially small measure near $z=0$ as the gamma distribution has exponential tails.

We reemphasize that our calculation of the local minima density was performed assuming the variance of the spectral distribution of each $\bm{O}^{\alpha}$ (in units of the mean eigenvalue) asymptotically vanishes. This was also the setting studied in previous work on the local minima of QNNs [, ]. This assumption allows us to dramatically simplify the Hessian to the form given in Theorem 19. Though it holds for low-weight fermionic [] and local spin Hamiltonians [], it does *not* hold for the Gaussian unitary ensemble (GUE) or nonlocal spin systems; these systems are also known to have efficient quantum algorithms that prepare their low-energy states, unlike their local cousins []. We hope in the future to analyze whether this property also has an impact on the behavior of local minima of QNNs.

We finally note an interesting connection between our results and algorithmic hardness. We here only calculate the expected density of local minima at a given function value. If the second moment of the local minima density is also sufficiently well-behaved, then it could be the case that the asymptotic density of local minima has a fixed distributional form; this is the case in (classical) spherical spin glass models []. The function value at which these local minima proliferate w.h.p. is conjectured to also hold as a *general* (i.e., beyond gradient descent) *algorithmic threshold*, that is, it is generically believed to be the function value at which better approximations to the ground state become algorithmically intractable to find. This conjecture is known to be true for the specific case of pure spherical spin glass models []. Studying second moments of quantum spin glass local minima distributions may thus be a tractable avenue for studying the algorithmic hardness of quantum problems.

###### Corollary 23 (Density of local minima to multiplicative leading order) .

###### Proof.

###### Corollary 24 (Density of local minima to multiplicative leading order, regularized) .

###### Proof.

## Appendix D Proofs of the Main Results

### D.1 Preliminaries

We now give in full detail the proofs of the main results discussed in Appendix B.2. We begin by giving definitions and notational conventions that we will use throughout our proofs. Recall that, given a JAWS $\mathcal{J}=\left(\bigoplus_{\alpha}\mathcal{A}_{\alpha},\mathcal{T},\bm{
\mathcal{G}},\bm{\mathcal{H}},\bm{A},\bm{O}\right)$, we are interested in the joint distribution of the loss $\ell\left(\bm{\theta};\bm{\rho}\right)$ and its first two derivatives over a set of input states $\bm{\rho}\in\mathcal{R}$, where:

$$ $\ell\left(\bm{\theta};\bm{\rho}\right)=\sum_{\alpha}\ell^{\alpha}\left(\bm{ \theta};\bm{\rho}\right)\equiv\sum_{\alpha}I_{\alpha}\Tr_{\alpha}\left(\bm{ \rho}^{\alpha}\bm{U}^{\alpha}\left(\bm{\theta}\right)\bm{O}^{\alpha}\bm{U}^{ \alpha\dagger}\left(\bm{\theta}\right)\right)$ (140) $$

and

$$ $\bm{U}^{\alpha}\left(\bm{\theta}\right)\equiv\bm{g}_{0}^{\alpha\dagger}\left( \prod_{i=1}^{p}\bm{g}_{i}^{\alpha}\exp\left(\theta_{i}\bm{A}_{i}^{\alpha} \right)\bm{g}_{i}^{\alpha\dagger}\right)\bm{h}^{\alpha}$ (141) $$

with $\bm{g}_{i}^{\alpha},\bm{h}^{\alpha}\in G_{\alpha}$, and $\bm{A}_{i}^{\alpha}\in\mathfrak{g}_{\alpha}$. As previously discussed we will use $\cdot^{\alpha}$ to denote the defining representation of the projection of $\cdot$ into $\mathcal{A}_{\alpha}$. As each sector labeled by $\alpha$ is independent we will here only consider a single $\alpha$ WLOG, with nonzero $\bm{\rho}^{\alpha}$ and $\bm{O}^{\alpha}$. This will also allow us to remove the cumbersome notation of labeling all objects with the index $\alpha$ for the remainder of this section. To further simplify the language, we will use the term “unitary” to refer to “orthogonal,” “unitary,” or “hyperunitary” in the context of $\mathbb{F}=\mathbb{R},\mathbb{C},\mathbb{H}$, respectively, unless otherwise explicitly stated. Similarly, we will later see that our results hold for any $\bm{\theta}$; we will thus leave the $\bm{\theta}$-dependence of $\ell$ implicit from here on out to save on notation. In the following we will use $\ket{\mu}$ (i.e., Greek letters), $0\leq\mu\leq N_{\alpha}-1$ to denote basis vectors in the vector space on which the defining representation of $\mathcal{H}^{N_{\alpha}}\left(\mathbb{F}_{\alpha}\right)$ acts. We will later use $\ket{i}$ (i.e., Latin letters), $0\leq i\leq p$ to denote vectors in the vector space on which the defining representation of $\mathcal{H}^{p+1}\left(\mathbb{F}_{\alpha}\right)$ acts.

We now detail our choice of $\bm{A}_{i}$ given Assumption 16. Assumption 16 has a nice interpretation as taking the $\bm{A}_{i}$ to be low-rank (representations of) basis elements of the Cartan subalgebra $\mathfrak{h}\subseteq\mathfrak{g}$. Up to the adjoint action of $G$ and an overall normalization (which can be absorbed into $\theta_{i}$), then, we can take WLOG:

$$ $\bm{A}_{i}=\ket{0}\bra{1}-\ket{1}\bra{0}$ (142) $$

when $\mathbb{F}=\mathbb{R}$ and

$$ $\bm{A}_{i}=\mathrm{i}\ket{0}\bra{0}$ (143) $$

when $\mathbb{F}=\mathbb{C}$. When $\mathbb{F}=\mathbb{H}$ we will take a parameterization of the form:

$$ $\exp\left(\theta_{i}\bm{A}_{i}\right)=\exp\left(\mathrm{i}\theta_{i}^{\left( \mathrm{i}\right)}\ket{0}\bra{0}\right)\exp\left(\mathrm{j}\theta_{i}^{\left( \mathrm{j}\right)}\ket{0}\bra{0}\right)\exp\left(\mathrm{k}\theta_{i}^{\left( \mathrm{k}\right)}\ket{0}\bra{0}\right);$ (144) $$

that is, we will assume we have full control over the quaternionic phase. We have chosen here for the $\bm{A}_{i}$ to be $i$-independent for convenience, moving any $i$-dependence to the conjugating unitaries of each layer $\bm{g}_{i}^{\alpha}\in G_{\alpha}$.

Finally, before continuing we give a convenient relation between

$$ $r=\frac{\left\lVert\bm{O}\right\rVert_{\ast}^{2}}{\left\lVert\bm{O}\right \rVert_{\text{F}}^{2}}$ (145) $$

and the standard deviation $\sigma_{o}$ of the eigenvalues of $\bm{O}$. This relation will be used to simplify some of our later expressions.

###### Lemma 25 ( $r$ and $\sigma_{o}$ relation) .

Let $\bm{O}$ be an $N\times N$ Hermitian operator and $r$ be as in Eq. (145). Let $\sigma_{o}$ be the standard deviation of the eigenvalues of $\bm{O}$, and $\overline{o}$ the arithmetic mean. Then:

$$ $\sqrt{\frac{N}{r}-1}=\frac{\sigma_{o}}{\overline{o}}.$ (146) $$

###### Proof.

Let $\overline{o}_{\textrm{RMS}}$ be the root mean square of the eigenvalues of $\bm{O}$. Then:

$$ $\displaystyle\sqrt{\frac{N}{r}-1}$ $\displaystyle=\sqrt{\frac{N\left\lVert\bm{O}\right\rVert_{\textrm{F}}^{2}}{ \left\lVert\bm{O}\right\rVert_{\ast}^{2}}-1}$ (147) $\displaystyle=\sqrt{\frac{\overline{o}_{\textrm{RMS}}^{2}}{\overline{o}^{2}}-1}$ $\displaystyle=\frac{\sigma_{o}}{\overline{o}}.$ $$

∎

###### Lemma 25 ( r 𝑟 r italic_r and σ o subscript 𝜎 𝑜 \sigma_{o} italic_σ start_POSTSUBSCRIPT italic_o end_POSTSUBSCRIPT relation) .

###### Proof.

### D.2 Asymptotic Expression for the Loss

We now proceed with the proofs of our main results. This subsection is devoted to a series of reductions that will allow us to consider the $\bm{h}\bm{O}\bm{h}^{\dagger}$ as $\beta$-Wishart matrices up to a controlled error in Lévy–Prokhorov metric, thus proving Theorem 13. Along the way we will also prove a reduction to taking the $\bm{g}_{i}$ to be Haar random, once again up to some bounded error in Lévy–Prokhorov metric. In proving these results we will heavily rely on various lemmas on convergence in distribution given in Appendix L.

#### D.2.1 Reduction to Haar Random 𝒉 , 𝒈 i 𝒉 subscript 𝒈 𝑖 \bm{h},\bm{g}_{i} bold_italic_h , bold_italic_g start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT

We first argue that, under Assumption 12, the $\bm{h}$ in Eq. (141) can be assumed to be Haar random over $G$ up to some bounded error in Lévy–Prokhorov metric. We also show that this remains true when scaling by any choice of $N$-dependent normalization $\mathcal{N}$ with the only requirement being that central moments of large (constant) order have a well-defined, finite limit as $N\to\infty$; for which maximal choice of $\mathcal{N}$ this is true depends on the exact form of the $\epsilon$-approximate $t$-design, but is always true for $\mathcal{N}=1$. In the following we implicitly consider a sequence of objects as $\epsilon^{-1},t,N\to\infty$.

###### Lemma 26 (Weak convergence to Haar random $\bm{h}$ ) .

Assume $\bm{h}$ forms an $\epsilon$-approximate $t$-design. Let $L_{i}$, $i\in\left[d\right]$ be multilinear functions of the form:

$$ $L_{i}=\Tr\left(\bm{M}_{i}\bm{h}\bm{O}\bm{h}^{\dagger}\right),$ (148) $$

where the $\bm{M}_{i}$ have bounded operator norm and $\bm{h}\sim\mathcal{H}$. Let $\tilde{L}_{i}$ be the same with $\bm{h}\to\bm{\tilde{h}}$, where $\bm{\tilde{h}}$ is Haar random. Assume any $N$-dependent normalization $\mathcal{N}\leq\operatorname{O}\left(N\right)$, and assume that there exists a constant $k^{\ast}$ such that all central moments of constant order $k>k^{\ast}$ of the $\mathcal{N}L_{i}$ have a finite limit as $\epsilon^{-1},t,N\to\infty$. Then the joint distribution of $\mathcal{N}L_{i}$ differs from that of $\mathcal{N}\tilde{L}_{i}$ by an error at most

$$ $\pi_{1}=\operatorname{O}\left(\frac{d\log\log\left(\mathcal{N}^{-t}\epsilon^{- 1}\right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+\frac{d\log\left(t \right)}{\sqrt{t}}\right)$ (149) $$

in Lévy–Prokhorov metric as $\epsilon^{-1},t,N\to\infty$.

###### Proof.

As the $L_{i}-\mathbb{E}\left[L_{i}\right]$ are bounded random variables, there exists some constant $C>0$ such that:

$$ $\mathbb{E}\left[\prod_{i\in s_{k}}\left(L_{i}-\mathbb{E}\left[L_{i}\right] \right)\right]\leq\left(C\sqrt{k}\right)^{k}$ (150) $$

for all index multisets $s_{k}$ of cardinality $k$. By the given assumption on $\mathcal{N}$ a similar bound also holds for central moments of the $\mathcal{N}L_{i}$ of sufficiently high order. The bound $\mathcal{N}=\operatorname{O}\left(N\right)$ ensures that this central moment bound also holds true for the $\mathcal{N}\tilde{L}_{i}$ by sub-Gaussianity [, ], and also ensures that the $\mu$ of Corollary 38 is subleading to Eq. (149). The conditions of Corollary 38 are then satisfied and the final result yielded.
∎

By incurring this error in Lévy–Prokhorov metric we now can assume that the $\bm{h}$ (given Assumption 12) are Haar random when considering just the distribution of the loss (so $d=1$ in Lemma 26). When considering the loss, gradient, and Hessian jointly, the addition of Assumption 15 means that we can assume that both the $\bm{h}$ and the $\bm{g}_{i}$ are Haar random. We note, however, that up to requiring a worse bound on $\mathcal{N}$ (i.e., $\mathcal{N}\leq\operatorname{o}\left(N^{\frac{2}{3}}\right)$) the $\bm{g}_{i}$ being drawn from an $\epsilon$-approximate $2$-design suffices. This argument is given in detail in Appendix I.

When the $\bm{h}$ and $\bm{g}_{i}$ are reduced to being drawn i.i.d. from the Haar distribution it is apparent that the joint distribution of the loss with its first two derivatives is invariant under translations of the parameters. This justifies us fixing $\bm{\theta}=\bm{0}$ in the sequel.

###### Lemma 26 (Weak convergence to Haar random 𝒉 𝒉 \bm{h} bold_italic_h ) .

###### Proof.

#### D.2.2 Reduction to Gaussian 𝒉 , 𝒈 i 𝒉 subscript 𝒈 𝑖 \bm{h},\bm{g}_{i} bold_italic_h , bold_italic_g start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT

We now show that certain marginal distributions of the the entries of $\sqrt{\mathcal{N}}\bm{h},\sqrt{\mathcal{N}}\bm{g}_{i}$ can be approximated as random Gaussian matrices over $\mathbb{F}$ up to a small error in Ky Fan metric whenever $\mathcal{N}\leq\operatorname{O}\left(N\right)$. This will allow us to replace $\sqrt{\mathcal{N}}\bm{h},\sqrt{\mathcal{N}}\bm{g}_{i}$ with random matrices with i.i.d. Gaussian entries, simplifying our results further.

###### Lemma 27 (Convergence in probability of Haar marginals to Gaussian matrices) .

Let $L_{i}$, $i\in\left[d\right]$ be uniformly bounded multilinear functions of the form:

$$ $L_{i}=\Tr\left(\bm{M}_{i}\bm{h}\bm{\rho}\bm{h}^{\dagger}\right),$ (151) $$

where there exists some $D$ such that

$$ $\frac{1}{\Tr\left(\bm{\rho}^{2}\right)}\leq D=\operatorname{o}\left(\frac{N}{ \log\left(N\right)}\right).$ (152) $$

Assume as well that $d\leq\operatorname{O}\left(\operatorname{poly}\left(N\right)\right)$ and the $\bm{M}_{i}$ are independent from the $\bm{h}$. Let $\tilde{L}_{i}$ be the same, where now the $\bm{h}$ are random matrices $\bm{G}$ with i.i.d. standard Gaussian entries over $\mathbb{F}$. For sufficiently large $N$, the joint distribution of $\mathcal{N}L_{i}$ differs from the joint distribution of $\frac{\mathcal{N}}{N}\tilde{L}_{i}$ by an error in Ky Fan metric of at most:

$$ $\alpha_{3}=\operatorname{O}\left(\frac{\sqrt{\mathcal{N}D\log\left(N\right)}}{ N}+\frac{\mathcal{N}}{D}\right)=\operatorname{o}\left(1\right)$ (153) $$

for all $\mathcal{N}\leq\operatorname{O}\left(N\right)$. Alternatively, if $\rank\left(\bm{\rho}\right)\leq D$,

$$ $\alpha_{3}=\operatorname{O}\left(\frac{\sqrt{\mathcal{N}D\log\left(N\right)}}{ N}\right)=\operatorname{o}\left(1\right).$ (154) $$

###### Proof.

We assume $\bm{\rho}$ has trace norm $1$ by absorbing $\Tr\left(\bm{\rho}\right)$ into $\bm{M}_{i}$. We first argue that if the high-purity condition is assumed then $\bm{\rho}$ has an approximation to small error in trace norm that is low-rank. To see this, let $r_{i}$ be the eigenvalues of $\bm{\rho}$ in non-increasing order. If

$$ $\sum_{i=D+1}^{N_{\alpha}}r_{i}\geq\frac{1}{D+1}$ (155) $$

then it must be that $\bm{\rho}$ has low purity, i.e.,

$$ $\Tr\left(\bm{\rho}^{2}\right)\leq\frac{D+1}{\left(D+1\right)^{2}}=\frac{1}{D+1 }<\frac{1}{D},$ (156) $$

breaking our assumption on the purities of the inputs (Eq. (152)). Thus,

$$ $\sum_{i=D+1}^{N_{\alpha}}r_{i}<\frac{1}{D+1}.$ (157) $$

In particular, $\bm{\rho}$ has a rank-$D$ approximation that agrees up to an $\operatorname{O}\left(\frac{1}{D}\right)$ additive error in trace distance.

Consider now the low-rank case. Let

$$ $\delta\equiv\frac{D\ln\left(N\right)}{N}.$ (158) $$

$\delta<\frac{1}{4}$ for sufficiently large $N$ by the assumed scaling of $D$. Following the proof of Corollary 1.1 in Jiang [], then, for sufficiently large $N$ and $t>10\sqrt{\frac{\mathcal{N}\delta}{N}}$,

$$ $\mathbb{P}\left[\max_{\begin{subarray}{c}i\in\left[N\right]\\ j\in\left[D\right]\end{subarray}}\left\lvert\sqrt{N}h_{i,j}-G_{i,j}\right \rvert\geq\sqrt{\frac{N}{\mathcal{N}}}t\right]=\exp\left(-\operatorname{\Omega }\left(\log\left(N\right)^{\frac{3}{2}}\right)\right)+\exp\left(-\operatorname {\Omega}\left(\frac{N^{2}}{\mathcal{N}D}\right)\right).$ (159) $$

The right-hand side decays superpolynomially with $N$ whenever $\mathcal{N}\leq\operatorname{O}\left(N\right)$. This implies the error in Ky Fan metric is dominated by the cutoff at $t=\operatorname{\Theta}\left(\sqrt{\frac{\mathcal{N}\delta}{N}}\right)$. This in conjunction with the error from the initial low-rank approximation yields the final result.
∎

We will mostly be concerned with the specific case when $D=N^{0.999}$, yielding for $\mathcal{N}\leq\operatorname{O}\left(N^{0.99}\right)$:

$$ $\alpha_{3}=\operatorname{O}\left(\sqrt{\frac{\mathcal{N}\log\left(N\right)}{N^ {1.001}}}\right)=\operatorname{o}\left(1\right).$ (160) $$

###### Lemma 27 (Convergence in probability of Haar marginals to Gaussian matrices) .

###### Proof.

#### D.2.3 Reduction to Semi-Isotropic 𝑶 𝑶 \bm{O} bold_italic_O

We end with a reduction to a more convenient form for the spectrum of $\bm{O}$. Let $\ket{\mu}$ be the eigenbasis of $\bm{O}$. Consider the Hermitian $\bm{\tilde{O}}$:

$$ $\bm{\tilde{O}}\equiv k\sum_{\mu=0}^{r-1}\ket{\mu}\bra{\mu},$ (161) $$

where

$$ $r\equiv\frac{\left\lVert\bm{O}\right\rVert_{\ast}^{2}}{\left\lVert\bm{O}\right \rVert_{\textrm{F}}^{2}}$ (162) $$

is assumed to be an integer and (for $\overline{o}$ the mean eigenvalue of $o$)

$$ $k\equiv\frac{\left\lVert\bm{O}\right\rVert_{\ast}}{r}=\frac{N\overline{o}}{r}.$ (163) $$

By construction

$$ $\displaystyle\Tr\left(\bm{O}\right)$ $\displaystyle=\Tr\left(\bm{\tilde{O}}\right),$ (164) $\displaystyle\Tr\left(\bm{O}^{2}\right)$ $\displaystyle=\Tr\left(\bm{\tilde{O}}^{2}\right).$ (165) $$

We claim that replacing $\bm{O}$ with $\bm{\tilde{O}}$ incurs only a vanishingly small error in Ky Fan metric. Intuitively this follows from the Welch–Satterthwaite approximation of a weighted sum of Wishart random matrices as a single Wishart random matrix []. We formally state this as the following lemma.

###### Lemma 28 (Reduction to semi-isotropic $\bm{O}$ ) .

Let $L_{i}$, $i\in\left[d\right]$ be of the form:

$$ $L_{i}=\Tr\left(\bm{M}_{i}\bm{X}\left(\bm{O}-\bm{\tilde{O}}\right)\bm{X}^{ \dagger}\right),$ (166) $$

where $\sqrt{N}\bm{X}$ has i.i.d. standard Gaussian entries over $\mathbb{F}$, $\bm{M}_{i}$ is independent from $\bm{X}$ with bounded trace norm, and $d=\operatorname{O}\left(\operatorname{poly}\left(N\right)\right)$. Let $\tilde{L}_{i}$ be the same multilinear functions, where instead of $\bm{O}$ one has $\bm{\tilde{O}}$ as defined in Eq. (161). The joint distribution of $\mathcal{N}L_{i}$ over $\bm{X}$ differs from $\bm{0}$ by an error at most

$$ $\alpha_{4}=\operatorname{O}\left(\frac{\mathcal{N}\varDelta\log\left(\frac{dN} {\mathcal{N}\varDelta}\right)}{N}\right)$ (167) $$

in Ky Fan metric, where

$$ $\varDelta\equiv 1-\frac{\overline{o}}{\left\lVert\bm{O}\right\rVert_{\text{op} }}-\frac{\sigma_{o}^{2}}{\left\lVert\bm{O}\right\rVert_{\text{op}}\overline{o}}.$ (168) $$

Here, $o_{i}$ is the $i$th eigenvalue of $\bm{O}$ in non-increasing order, $\sigma_{o}$ the standard deviation of the eigenvalues of $\bm{O}$, and $\overline{o}$ the mean.

###### Proof.

$\bm{O}$ and $\bm{\tilde{O}}$ can be assumed to be diagonal WLOG as $\bm{X}$ is unitarily invariant. Therefore, $\bm{X}\left(\bm{O}-\bm{\tilde{O}}\right)\bm{X}^{\dagger}$ can be written as a weighted sum of standard $\beta$-Wishart matrices each with a single degree of freedom:

$$ $\bm{X}\left(\bm{O}-\bm{\tilde{O}}\right)\bm{X}^{\dagger}=\frac{1}{N}\sum_{i=1} ^{N}\left(o_{i}-\tilde{o}_{i}\right)\ket{x_{i}}\bra{x_{i}},$ (169) $$

where $\ket{x_{i}}$ is the (unnormalized) $i$th column of $\sqrt{N}\bm{X}$ and $o_{i},\tilde{o}_{i}$ are the eigenvalues of $\bm{O},\bm{\tilde{O}}$, respectively, in nonincreasing order. Note that for any normalized $\ket{\mu}$, $\left\lvert\bra{x_{i}}\ket{\mu}\right\rvert$ is gamma-distributed. Thus, from a generalization of Bernstein’s inequality (see, e.g., Theorem 2.8.1 of Vershynin []), for any $\ket{\mu}$,

$$ $\displaystyle\mathbb{P}$ $\displaystyle\left[\left\lvert\frac{1}{N}\sum_{i=1}^{N}\left(o_{i}-\tilde{o}_{ i}\right)\left\lvert\bra{x_{i}}\ket{\mu}\right\rvert^{2}\right\rvert\geq \epsilon\right]$ (170) $\displaystyle\leq 2\exp\left(-cN\min\left(\frac{\epsilon}{\max_{i}\left\lvert o _{i}-\tilde{o}_{i}\right\rvert},\frac{\epsilon^{2}}{\frac{1}{N}\sum_{i=1}^{N} \left(o_{i}-\tilde{o}_{i}\right)^{2}}\right)\right)$ $$

for some constant $c>0$. Due to the equality of traces of $\bm{O}$ and $\bm{\tilde{O}}$ we calculate:

$$ $\displaystyle\frac{1}{N}\sum_{i=1}^{N}\left(o_{i}-\tilde{o}_{i}\right)^{2}$ $\displaystyle=\frac{1}{N}\Tr\left(\left(\bm{O}-\bm{\tilde{O}}\right)^{2}\right)$ (171) $\displaystyle=\frac{2}{N}\left(\Tr\left(\bm{O}^{2}\right)-\Tr\left(\bm{O}\bm{ \tilde{O}}\right)\right)$ $\displaystyle\leq\frac{2}{N}\left(\Tr\left(\bm{O}^{2}\right)-\Tr\left(\bm{O} \right)k+ko_{r+1}\right)$ $\displaystyle=\frac{2}{r}\overline{o}o_{r+1}$ $\displaystyle=\operatorname{O}\left(\frac{o_{r+1}}{N}\right).$ $$

Furthermore,

$$ $\displaystyle\max_{i}\left\lvert o_{i}-\tilde{o}_{i}\right\rvert$ $\displaystyle=\operatorname{O}\left(o_{1}-k\right)+\operatorname{O}\left(o_{r+ 1}\right)$ (172) $\displaystyle=\operatorname{O}\left(o_{1}-\overline{o}-\frac{\sigma_{o}^{2}}{ \overline{o}}+o_{r+1}\right)$ $\displaystyle=\operatorname{O}\left(1-\frac{\overline{o}}{\left\lVert\bm{O} \right\rVert_{\text{op}}}-\frac{\sigma_{o}^{2}}{\left\lVert\bm{O}\right\rVert_ {\text{op}}\overline{o}}\right).$ $$

The desired convergence follows by the union bound and taking $\epsilon\to\frac{\epsilon}{\mathcal{N}}$.
∎

We now claim these suffice to prove Theorem 13.

###### Proof of Theorem 13 .

For $\bm{X}$ with i.i.d. standard Gaussian entries over $\mathbb{F}$ and $\bm{\tilde{O}}$ as in Lemma 28, $k^{-1}\bm{X}\bm{\tilde{O}}\bm{X}^{\dagger}$ is $\beta$-Wishart-distributed with $r$ degrees of freedom. The result then follows from Lemmas 26, 27, and 28 applied in sequence, with a final error in Lévy–Prokhorov metric of:

$$ $\pi=\pi_{1}+\alpha_{3}+\alpha_{4}=\operatorname{O}\left(\frac{\log\log\left( \mathcal{N}^{-t}\epsilon^{-1}\right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1} \right)}+\frac{\log\left(t\right)}{\sqrt{t}}+\sqrt{\frac{\mathcal{N}\log\left( N\right)}{N^{1.001}}}\right).$ (173) $$

∎

###### Lemma 28 (Reduction to semi-isotropic 𝑶 𝑶 \bm{O} bold_italic_O ) .

###### Proof.

###### Proof of Theorem 13 .

### D.3 Asymptotic Expressions for the First Two Derivatives

We will now move on and construct an explicit asymptotic expression for the first two derivatives of the loss function. As previously mentioned, we need only consider $\bm{\theta}=\bm{0}$ by Lemma 26. Here we have that the first derivatives are distributed as:

$$ $\partial_{i}\ell=I\Tr\left(\bm{\rho}\left[\bm{g}_{i}\bm{A}_{i}\bm{g}_{i}^{ \dagger},\bm{h}\bm{O}\bm{h}^{\dagger}\right]\right).$ (174) $$

The second derivatives are of the form (for $i\geq j$):

$$ $\partial_{i}\partial_{j}\ell=I\Tr\left(\bm{\rho}\left[\bm{g}_{j}\bm{A}_{j}\bm{ g}_{j}^{\dagger}\left[\bm{g}_{i}\bm{A}_{i}\bm{g}_{i}^{\dagger},\bm{h}\bm{O}\bm {h}^{\dagger}\right]\right]\right).$ (175) $$

Furthermore, using the lemmas from Appendix D.2, up to a cost:

$$ $\pi_{1}+\alpha_{3}+\alpha_{4}=\operatorname{O}\left(\frac{d\log\log\left( \mathcal{N}^{-t}\epsilon^{-1}\right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1} \right)}+\frac{d\log\left(t\right)}{\sqrt{t}}+\sqrt{\frac{\mathcal{N}\log\left (N\right)}{N}}\right)$ (176) $$

in Lévy–Prokhorov metric—where $d$ is either $p$ or $p^{2}$—we may reduce to considering:

$$ $\displaystyle\hat{\ell}$ $\displaystyle=\frac{I}{N}\Tr\left(\bm{\rho}\bm{X}_{0}^{\dagger}\bm{O}\bm{X}_{0 }\right),$ (177) $\displaystyle\hat{\ell}_{;i}$ $\displaystyle=\frac{I}{N^{2}}\Tr\left(\bm{\rho}\bm{X}_{0}^{\dagger}\left[\bm{X }_{i}\bm{A}_{i}\bm{X}_{i}^{\dagger},\bm{O}\right]\bm{X}_{0}\right),$ (178) $\displaystyle\hat{\ell}_{;i,j}$ $\displaystyle=\frac{I}{N^{3}}\Tr\left(\bm{\rho}\bm{X}_{0}^{\dagger}\left[\bm{X }_{j}\bm{A}_{j}\bm{X}_{j}^{\dagger},\left[\bm{X}_{i}\bm{A}_{i}\bm{X}_{i}^{ \dagger},\bm{O}\right]\right]\bm{X}_{0}\right),$ (179) $$

where:

- 1.
$\bm{X}_{0}$ is an $N\times\rank\left(\bm{\rho}\right)$ random matrix with i.i.d. standard Gaussian entries over $\mathbb{F}$, and the $\bm{X}_{i}$ are $N\times p$ or $N\times 2p$ (depending on the rank of the $\bm{A}_{i}$) random matrices with i.i.d. standard Gaussian entries over $\mathbb{F}$.
- 2.
The initial states $\bm{\rho}$ are of rank at most $\operatorname{O}\left(N^{0.999}\right)$.

Note that these quantities depend only on $M=\operatorname{O}\left(\rank\left(\bm{\rho}\right)\right)$ rows of $\bm{X}_{0}$ and either $1$ ($\mathbb{F}=\mathbb{C},\mathbb{H}$) or $2$ ($\mathbb{F}=\mathbb{R}$) rows of $\bm{X}_{i}$. We can therefore further simplify this by collecting all of the relevant rows of $\bm{X}_{0},\bm{X}_{i}$ into a single $\bm{X}$ of dimensions $N\times\left(p+M\right)$ (or $N\times\left(2p+M\right)$ when $\mathbb{F}=\mathbb{R}$), writing these expressions as(^8^88We have slightly abused notation here; the $\ket{i},\ket{j}$ here are taken to be orthogonal to the nonzero eigenvectors of $\bm{\rho}$.):

$$ $\displaystyle\hat{\ell}$ $\displaystyle=\frac{I\overline{o}}{r}\Tr\left(\bm{\rho}\bm{X}^{\dagger}\bm{ \tilde{O}}\bm{X}\right),$ (180) $\displaystyle\hat{\ell}_{;i}$ $\displaystyle=\frac{I\overline{o}}{rN}\Tr\left(\bm{\rho}\bm{X}^{\dagger}\left[ \bm{X}\ket{i}\bra{i}\bm{X}^{\dagger},\bm{\tilde{O}}\right]\bm{X}\right),$ (181) $\displaystyle\hat{\ell}_{;i,j}$ $\displaystyle=\frac{I\overline{o}}{rN^{2}}\Tr\left(\bm{\rho}\bm{X}^{\dagger} \left[\bm{X}\ket{j}\bra{j}\bm{X}^{\dagger},\left[\bm{X}\ket{i}\bra{i}\bm{X}^{ \dagger},\bm{\tilde{O}}\right]\right]\bm{X}\right).$ (182) $$

Here, we took another error of $\alpha_{4}$ in Lévy–Prokhorov metric to change $\bm{O}$ to

$$ $\bm{\tilde{O}}=\sum_{\mu=0}^{r-1}\ket{\mu}\bra{\mu}$ (183) $$

as in Eq. (161). Though this expression for the full distribution is unwieldy, it completely characterizes the joint distribution of the loss and first two derivatives in terms of a single Gaussian random matrix $\bm{X}$. To simplify further we will assume $\bm{\rho}$ is a single state which is rank-$1$ when projected into the simple sector we are considering, taken WLOG to be $\ket{0}\bra{0}$. We will also assume in the following that $\frac{\sigma_{o}}{\overline{o}}$ is asymptotically bounded.

Details of further simplifications vary depending on whether $\mathbb{F}$ is algebraically closed or not (i.e., whether or not we are working in $\mathbb{R}$). We will thus consider the two cases separately.

#### D.3.1 𝔽 = ℂ , ℍ 𝔽 ℂ ℍ \mathbb{F}=\mathbb{C},\mathbb{H} blackboard_F = blackboard_C , blackboard_H

When $\mathbb{F}=\mathbb{C},\mathbb{H}$, we are considering the joint distribution of:

$$ $\displaystyle\hat{\ell}$ $\displaystyle=\frac{I\overline{o}}{r}\Tr\left(\ket{0}\bra{0}\bm{X}^{\dagger} \bm{\tilde{O}}\bm{X}\right),$ (184) $\displaystyle\hat{\ell}_{;i}$ $\displaystyle=\frac{\mathrm{i}I\overline{o}}{rN}\Tr\left(\ket{0}\bra{0}\bm{X}^ {\dagger}\left[\bm{X}\ket{i}\bra{i}\bm{X}^{\dagger},\bm{\tilde{O}}\right]\bm{X }\right),$ (185) $\displaystyle\hat{\ell}_{;i,j}$ $\displaystyle=-\frac{I\overline{o}}{rN^{2}}\Tr\left(\ket{0}\bra{0}\bm{X}^{ \dagger}\left[\bm{X}\ket{j}\bra{j}\bm{X}^{\dagger},\left[\bm{X}\ket{i}\bra{i} \bm{X}^{\dagger},\bm{\tilde{O}}\right]\right]\bm{X}\right),$ (186) $$

where now $\bm{X}$ is a $N\times\left(p+1\right)$ random matrix with i.i.d. standard Gaussian entries over $\mathbb{F}$. Here, when $\mathbb{F}=\mathbb{H}$, “$\mathrm{i}$” should be taken to mean one of $\mathrm{i},\mathrm{j},\mathrm{k}$, depending on which parameter of Eq. (144) the derivative is being taken with respect to. We take the two contributions of quaternionic phase to be equal in the second derivative expression as, by the anticommutivity of quaternions, this expression is otherwise identically zero. In particular, this abuse of notation will not change any of the following analysis unless specifically stated otherwise.

Recall that

$$ $\bm{\tilde{O}}=\sum_{\mu=0}^{r-1}\ket{\mu}\bra{\mu}.$ (187) $$

Let $\bm{C}$ be the complement of $\bm{\tilde{O}}$, i.e.,

$$ $\bm{C}=\sum_{\mu=r}^{N-1}\ket{\mu}\bra{\mu}.$ (188) $$

In particular, $\bm{C}$ and $\bm{\tilde{O}}$ sum to the identity:

$$ $\bm{C}+\bm{\tilde{O}}=\bm{I}_{N}.$ (189) $$

Furthermore,

$$ $\displaystyle\bm{W}$ $\displaystyle\equiv\bm{X}^{\dagger}\bm{\tilde{O}}\bm{X},$ (190) $\displaystyle\bm{V}$ $\displaystyle\equiv\bm{X}^{\dagger}\bm{C}\bm{X},$ (191) $$

are each $\beta$-Wishart-distributed matrices; the former with $r$ degrees of freedom, and the latter with $N-r$. The independence of $\bm{W}$ and $\bm{V}$ can be checked from the associated Bartlett decompositions. We thus have that the distribution we are interested in is the joint distribution of:

$$ $\displaystyle\hat{\ell}$ $\displaystyle=\frac{I\overline{o}}{r}\bra{0}\bm{W}\ket{0},$ (192) $\displaystyle\hat{\ell}_{;i}$ $\displaystyle=\frac{2I\overline{o}}{rN}\real\left\{\mathrm{i}\bra{0}\bm{W}\ket {i}\bra{i}\bm{X}^{\dagger}\bm{X}\ket{0}\right\}=\frac{2I\overline{o}}{rN}\real \left\{\mathrm{i}\bra{0}\bm{W}\ket{i}\bra{i}\bm{V}\ket{0}\right\},$ (193) $$

and:

$$ $\displaystyle\hat{\ell}_{;i,j}=$ $\displaystyle-\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\dagger}\left[\bm{X} \ket{j}\bra{j}\bm{X}^{\dagger},\left[\bm{X}\ket{i}\bra{i}\bm{X}^{\dagger},\bm{ O}\right]\right]\bm{X}\ket{0}$ (194) $\displaystyle=$ $\displaystyle-\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\dagger}\bm{X}\ket{j} \bra{j}\bm{X}^{\dagger}\bm{X}\ket{i}\bra{i}\bm{X}^{\dagger}\bm{O}\bm{X}\ket{0}$ $\displaystyle-\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\dagger}\bm{O}\bm{X} \ket{i}\bra{i}\bm{X}^{\dagger}\bm{X}\ket{j}\bra{j}\bm{X}^{\dagger}\bm{X}\ket{0}$ $\displaystyle+\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\dagger}\bm{X}\ket{i} \bra{i}\bm{X}^{\dagger}\bm{O}\bm{X}\ket{j}\bra{j}\bm{X}^{\dagger}\bm{X}\ket{0}$ $\displaystyle+\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\dagger}\bm{X}\ket{j} \bra{j}\bm{X}^{\dagger}\bm{O}\bm{X}\ket{i}\bra{i}\bm{X}^{\dagger}\bm{X}\ket{0}$ $\displaystyle=$ $\displaystyle-\frac{I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \ket{j}\bra{j}\left(\bm{W}+\bm{V}\right)\ket{i}\bra{i}\bm{W}\ket{0}$ $\displaystyle-\frac{I\overline{o}}{rN^{2}}\bra{0}\bm{W}\ket{i}\bra{i}\left(\bm {W}+\bm{V}\right)\ket{j}\bra{j}\left(\bm{W}+\bm{V}\right)\ket{0}$ $\displaystyle+\frac{I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \ket{i}\bra{i}\bm{W}\ket{j}\bra{j}\left(\bm{W}+\bm{V}\right)\ket{0}$ $\displaystyle+\frac{I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \ket{j}\bra{j}\bm{W}\ket{i}\bra{i}\left(\bm{W}+\bm{V}\right)\ket{0}.$ $$

We can simplify the expression for the second derivative by grouping terms by their order in $\bm{W}$. Note first that all terms cubic in matrix elements of $\bm{W}$ sum to zero. For the quadratic terms, note that for the outer two terms:

$$ $\displaystyle-\bra{0}\bm{V}\ket{j}\bra{j}\bm{W}\ket{i}\bra{i}\bm{W}\ket{0}- \bra{0}\bm{W}\ket{j}\bra{j}\bm{V}\ket{i}\bra{i}\bm{W}\ket{0}$ (195) $\displaystyle+\bra{0}\bm{V}\ket{j}\bra{j}\bm{W}\ket{i}\bra{i}\bm{W}\ket{0}+ \bra{0}\bm{W}\ket{j}\bra{j}\bm{W}\ket{i}\bra{i}\bm{V}\ket{0}$ $\displaystyle=$ $\displaystyle-\bra{0}\bm{W}\ket{j}\bra{j}\bm{V}\ket{i}\bra{i}\bm{W}\ket{0}+ \bra{0}\bm{W}\ket{j}\bra{j}\bm{W}\ket{i}\bra{i}\bm{V}\ket{0},$ $$

and for the inner two terms:

$$ $\displaystyle-\bra{0}\bm{W}\ket{i}\bra{i}\bm{V}\ket{j}\bra{j}\bm{W}\ket{0}- \bra{0}\bm{W}\ket{i}\bra{i}\bm{W}\ket{j}\bra{j}\bm{V}\ket{0}$ (196) $\displaystyle+\bra{0}\bm{V}\ket{i}\bra{i}\bm{W}\ket{j}\bra{j}\bm{W}\ket{0}+ \bra{0}\bm{W}\ket{i}\bra{i}\bm{W}\ket{j}\bra{j}\bm{V}\ket{0}$ $\displaystyle=$ $\displaystyle-\bra{0}\bm{W}\ket{i}\bra{i}\bm{V}\ket{j}\bra{j}\bm{W}\ket{0}+ \bra{0}\bm{V}\ket{i}\bra{i}\bm{W}\ket{j}\bra{j}\bm{W}\ket{0}.$ $$

We can then combine terms (along with the terms linear in $\bm{W}$) to see that:

$$ $\displaystyle\hat{\ell}_{;i,j}=$ $\displaystyle\frac{2I\overline{o}}{rN^{2}}\real\left\{\bra{0}\bm{V}\ket{i}\bra {i}\bm{W}\ket{j}\bra{j}\left(\bm{W}+\bm{V}\right)\ket{0}\right\}$ (197) $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\real\left\{\bra{0}\bm{W}\ket{i} \bra{i}\bm{V}\ket{j}\bra{j}\left(\bm{W}+\bm{V}\right)\ket{0}\right\}.$ $$

We can simplify the Hessian even further if we assume that $\frac{\sigma_{o}}{\overline{o}}=\operatorname{o}\left(1\right)$ by dropping terms that are w.h.p. subleading in $\frac{\sigma_{o}}{\overline{o}}$. In particular, consider the following terms:

$$ $\displaystyle\begin{aligned} \frac{1}{rN}\real&\left\{\bra{0}\bm{V}\ket{i}\bra {i}\bm{W}\ket{j}\bra{j}\bm{V}\ket{0}\right\}\\ &\leq\frac{N-r}{N}\left\lvert\frac{1}{\sqrt{N-r}}\bra{0}\bm{V}\ket{i}\right \rvert\left\lvert\frac{1}{r}\bra{i}\bm{W}\ket{j}\right\rvert\left\lvert\frac{1 }{\sqrt{N-r}}\bra{j}\bm{V}\ket{0}\right\rvert,\end{aligned}$ (198) $\displaystyle\begin{aligned} \frac{1}{rN}\real&\left\{\bra{0}\bm{W}\ket{i}\bra {i}\bm{V}\ket{j}\bra{j}\bm{W}\ket{0}\right\}\\ &\leq\frac{N-r}{N}\left\lvert\frac{1}{\sqrt{r}}\bra{0}\bm{W}\ket{i}\right \rvert\left\lvert\frac{1}{N-r}\bra{i}\bm{V}\ket{j}\right\rvert\left\lvert\frac {1}{\sqrt{r}}\bra{j}\bm{W}\ket{0}\right\rvert,\end{aligned}$ (199) $\displaystyle\begin{aligned} \frac{1}{rN}\real&\left\{\bra{0}\bm{W}\ket{i}\bra {i}\bm{V}\ket{j}\bra{j}\bm{V}\ket{0}\right\}\\ &\leq\frac{\left(N-r\right)^{\frac{3}{2}}}{\sqrt{r}N}\left\lvert\frac{1}{\sqrt {r}}\bra{0}\bm{W}\ket{i}\right\rvert\left\lvert\frac{1}{N-r}\bra{i}\bm{V}\ket{ j}\right\rvert\left\lvert\frac{1}{\sqrt{N-r}}\bra{j}\bm{V}\ket{0}\right\rvert. \end{aligned}$ (200) $$

Recall that the diagonal elements of a $\beta$-Wishart matrix are $\chi^{2}$-distributed with $\beta r$ degrees of freedom (see Appendix A.4) and thus have exponential tails. In Lemma 43—proven in Appendix M—we prove that the off-diagonal elements also have exponential tails. In particular, all three of these terms are of the form $\kappa R_{1}R_{2}R_{3}$, where $0<\kappa<1$ is some overall prefactor and the $R_{i}$ are random variables obeying tail bounds of the form:

$$ $\mathbb{P}\left[R_{i}\geq 1+t\right]\leq K\exp\left(-C\min\left(t,t^{2}\right)\right)$ (201) $$

for some universal constants $C,K>0$. We thus have by the union bound:

$$ $\displaystyle\mathbb{P}$ $\displaystyle\left[\kappa R_{1}R_{2}R_{3}\geq\left(\kappa^{\frac{1}{3}}+t \right)^{3}\right]$ (202) $\displaystyle\leq\mathbb{P}\left[\kappa\max\left(R_{1},R_{2},R_{3}\right)^{3} \geq\left(\kappa^{\frac{1}{3}}+t\right)^{3}\right]$ $\displaystyle\leq 3\max\left(\mathbb{P}\left[\kappa^{\frac{1}{3}}R_{1}\geq \kappa^{\frac{1}{3}}+t\right],\mathbb{P}\left[\kappa^{\frac{1}{3}}R_{2}\geq \kappa^{\frac{1}{3}}+t\right],\mathbb{P}\left[\kappa^{\frac{1}{3}}R_{3}\geq \kappa^{\frac{1}{3}}+t\right]\right)$ $\displaystyle\leq 3K\exp\left(-C\min\left(\kappa^{-\frac{1}{3}}t,\kappa^{- \frac{2}{3}}t^{2}\right)\right).$ $$

Taking into account the $p^{2}$ matrix elements via the union bound, these terms thus converge to 0 0 in Ky Fan metric at a rate $\operatorname{\Omega}\left(\kappa^{\frac{1}{3}}\log\left(\kappa^{-\frac{1}{3}}
p\right)\right)$. We can similarly use Lemma 41 to justify replacing $\bm{W}$ with a low-rank perturbation $\bm{\tilde{W}}$, which differs only in the removal of the first column of its Bartlett factor $\bm{L}_{W}$. Altogether, we have that we can take up to an order

$$ $\alpha_{5}=\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}\left(N-r \right)^{\frac{1}{3}}\log\left(\frac{p^{2}N^{\frac{2}{3}}}{\mathcal{N}^{\frac{ 1}{3}}\left(N-r\right)^{\frac{1}{3}}}\right)}{N^{\frac{2}{3}}}\right)\leq \operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}\sigma_{o}^{\frac{2}{3}} \log\left(\frac{p^{2}N^{\frac{1}{3}}\overline{o}^{\frac{2}{3}}}{\mathcal{N}^{ \frac{1}{3}}\sigma_{o}^{\frac{2}{3}}}\right)}{N^{\frac{1}{3}}\overline{o}^{ \frac{2}{3}}}\right)$ (203) $$

error in Ky Fan metric:

$$ $\hat{\ell}_{;i,j}=\frac{2I\overline{o}}{rN^{2}}\real\left\{\bra{0}\bm{V}\ket{i }\bra{i}\bm{\tilde{W}}\ket{j}\bra{j}\bm{W}\ket{0}\right\}.$ (204) $$

At this point it is instructive to consider this joint distribution in terms of the marginal distributions of elements in the Bartlett decompositions of $\bm{W}$ and $\bm{V}$. Let $N_{W,i,j}$ be the i.i.d. standard normally distributed random variables over $\mathbb{F}$ in the off-diagonal elements of $\bm{L}_{W}$ (with the convention $i<j$), and similarly for $N_{V,i,j}$ and the Bartlett factor $\bm{L}_{V}$ of $\bm{V}$. Furthermore, let $\beta^{-\frac{1}{2}}\chi_{W,i}$ be the i.i.d. $\chi$-distributed random variables along the diagonal of $\bm{L}_{W}$, and similarly for $\beta^{-\frac{1}{2}}\chi_{V,i}$ and $\bm{L}_{V}$.

We first condition on $\hat{\ell}=z$. From Eq. (192) this is equivalent to conditioning on $\chi_{W,0}=\sqrt{\frac{\beta r}{I\overline{o}}z}$. Conditioned on this event, the first derivative is distributed as:

$$ $\hat{\ell}_{;i\mid z}=\frac{2I\beta\overline{o}}{rN}\sqrt{\frac{r}{I\overline{ o}}z}\chi_{V,0}\real\left\{\mathrm{i}N_{W,0,i}^{\ast}N_{V,0,i}\right\}.$ (205) $$

Similarly, the second derivative conditioned on this event is distributed as:

$$ $\hat{\ell}_{;i,j\mid z}=\frac{2I\beta\overline{o}}{rN^{2}}\sqrt{\frac{r}{I \overline{o}}z}\chi_{V,0}\real\left\{N_{V,0,i}^{\ast}\bra{i}\bm{\tilde{W}}\ket {j}N_{W,0,j}\right\}.$ (206) $$

Note that $\bm{\tilde{W}}$ is independent from $N_{W,0,i}$ exactly due to the removal of the first column of the Bartlett factor of $\bm{W}$.

We will next simplify further using the fact that $\chi_{V,0}$ concentrates around its mean, in particular using the $\chi$-distribution tail bound (for $N-r>0$, some universal constant $C>0$, and bounded $t$):

$$ $\mathbb{P}\left[\frac{1}{\left(N-r\right)^{\frac{1}{4}}}\left\lvert\chi-\sqrt{ \beta\left(N-r\right)}\right\rvert\geq t\right]\leq 2\exp\left(-C\sqrt{N-r} \min\left(t,t^{2}\right)\right)$ (207) $$

which follows from Lemma 42, proved in Appendix M. Just as we accounted for $\alpha_{5}$, up to an order

$$ $\alpha_{6}=\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}r^{\frac{1}{6} }\left(N-r\right)^{\frac{1}{12}}\log\left(\frac{p^{2}N^{\frac{2}{3}}}{\mathcal {N}^{\frac{1}{3}}r^{\frac{1}{6}}\left(N-r\right)^{\frac{1}{12}}}\right)}{N^{ \frac{2}{3}}}\right)\leq\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}} \sigma_{o}^{\frac{1}{6}}\log\left(\frac{p^{2}N^{\frac{5}{12}}\overline{o}^{ \frac{1}{6}}}{\mathcal{N}^{\frac{1}{3}}\sigma_{o}^{\frac{1}{6}}}\right)}{N^{ \frac{5}{12}}\overline{o}^{\frac{1}{6}}}\right)$ (208) $$

error in Ky Fan metric,

$$ $\displaystyle\hat{\ell}_{;i\mid z}$ $\displaystyle=\frac{2I\beta\overline{o}}{N}\sqrt{\frac{\beta}{I\overline{o}} \left(\frac{N}{r}-1\right)z}\real\left\{\mathrm{i}N_{W,0,i}^{\ast}N_{V,0,i} \right\},$ (209) $\displaystyle\hat{\ell}_{;i,j\mid z}$ $\displaystyle=\frac{2I\beta\overline{o}}{N^{2}}\sqrt{\frac{\beta}{I\overline{o }}\left(\frac{N}{r}-1\right)z}\real\left\{N_{V,0,i}^{\ast}\bra{i}\bm{\tilde{W} }\ket{j}N_{W,0,j}\right\}.$ (210) $$

Using Lemma 25 we can simplify this further as:

$$ $\displaystyle\hat{\ell}_{;i\mid z}$ $\displaystyle=\frac{2I\beta\sigma_{o}}{N}\sqrt{\frac{\beta z}{I\overline{o}}} \real\left\{\mathrm{i}N_{W,0,i}^{\ast}N_{V,0,i}\right\},$ (211) $\displaystyle\hat{\ell}_{;i,j\mid z}$ $\displaystyle=\frac{2I\beta\sigma_{o}}{N^{2}}\sqrt{\frac{\beta z}{I\overline{o }}}\real\left\{N_{V,0,i}^{\ast}\bra{i}\bm{\tilde{W}}\ket{j}N_{W,0,j}\right\}.$ (212) $$

We can simplify this even further by realizing that the distribution of $N_{V,0,i}$ is invariant under multiplication by an overall phase, i.e., it is circularly symmetric. We can thus absorb the argument of $N_{W,0,i}^{\ast}$ into $N_{V,0,i}$; we simultaneously conjugate $\bm{\tilde{W}}$ by unitary operators corresponding to this phase, so we need only consider the joint distribution of:

$$ $\displaystyle\hat{\ell}_{;i\mid z}$ $\displaystyle=\frac{2I\beta\sigma_{o}}{N}\sqrt{\frac{z}{I\overline{o}}}\chi_{i }\real\left\{\mathrm{i}N_{V,0,i}\right\},$ (213) $\displaystyle\hat{\ell}_{;i,j\mid z}$ $\displaystyle=\frac{2I\beta\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I\overline{o}}} \chi_{j}\bra{i}\real\left\{N_{V,0,i}^{\ast}\bm{\tilde{W}}\right\}\ket{j},$ (214) $$

where here the $\chi_{i}$ are i.i.d. $\chi$-distributed random variables with $\beta$ degrees of freedom given by:

$$ $\chi_{i}\equiv\sqrt{\beta}\left\lvert N_{W,0,i}\right\rvert.$ (215) $$

Noting $\sqrt{\beta}\real\left\{\mathrm{i}N_{V,0,i}\right\}$ is distributed as a standard normal random variable proves Theorem 17 for sectors with $\mathbb{F}=\mathbb{C},\mathbb{H}$, with accumulated error in Lévy–Prokhorov distance:

$$ $\pi^{\prime}=\pi_{1}+\alpha_{3}+2\alpha_{4}+\alpha_{6}=\operatorname{O}\left( \frac{p\log\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}{\log\left(\mathcal{ N}^{-t}\epsilon^{-1}\right)}+\frac{p\log\left(t\right)}{\sqrt{t}}+\sqrt{\frac{ \mathcal{N}\log\left(N\right)}{N^{1.001}}}+\frac{\mathcal{N}^{\frac{1}{3}}\log \left(N\right)}{N^{\frac{5}{12}}}\right).$ (216) $$

We finally consider the second derivative conditioned on both $\hat{\ell}=z$ and all $\hat{\ell}_{;i\mid z}=0$ (for $z>0$). Note that the $\chi$ distribution density is zero at 0 0. In particular, this conditioning is equivalent to conditioning on $\real\left\{\mathrm{i}N_{V,0,i}\right\}=0$. Furthermore, when $\mathbb{F}=\mathbb{H}$, recall that “$\mathrm{i}$” could be any one of $\mathrm{i},\mathrm{j},\mathrm{k}$ due to Eq. (144). In particular, this is conditioning on *all* nonreal components of $N_{V,0,i}$ equaling zero. Taking this all together yields:

$$ $\hat{\ell}_{;i,j\mid z,\bm{0}}=\frac{2I\beta\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I \overline{o}}}\chi_{j}\real\left\{N_{V,0,i}^{\ast}\right\}\bra{i}\real\left\{ \bm{\tilde{W}}\right\}\ket{j}.$ (217) $$

As the real parts of complex or symplectic Wishart matrices are real Wishart with a factor of $\beta$ more degrees of freedom and rescaled by a factor of $\beta^{-1}$, and similarly the real parts of standard complex or symplectic Gaussian random variables are standard real Gaussian random variables rescaled by a factor of $\beta^{-\frac{1}{2}}$, we can more conveniently rewrite this as

$$ $\hat{\ell}_{;i,j\mid z,\bm{0}}=\frac{2I\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I\beta \overline{o}}}\hat{N}_{i}\chi_{j}\bra{i}\bm{\hat{W}}\ket{j},$ (218) $$

where now $\bm{\hat{W}}$ is a real Wishart matrix with $\beta r$ degrees of freedom and the $\hat{N}_{i}$ are i.i.d. standard normal random variables. This proves Theorem 19 for sectors with $\mathbb{F}=\mathbb{C},\mathbb{H}$, with accumulated error in Lévy–Prokhorov distance:

$$ $\pi^{\prime\prime}=\pi_{1}+\alpha_{3}+2\alpha_{4}+\alpha_{5}+\alpha_{6}= \operatorname{O}\left(\frac{p^{2}\log\log\left(\mathcal{N}^{-t}\epsilon^{-1} \right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+\frac{p^{2}\log\left( t\right)}{\sqrt{t}}+\frac{\mathcal{N}^{\frac{1}{3}}\log\left(N\right)}{N^{ \frac{1}{3}}}\right).$ (219) $$

#### D.3.2 𝔽 = ℝ 𝔽 ℝ \mathbb{F}=\mathbb{R} blackboard_F = blackboard_R

We now consider when $\mathbb{F}=\mathbb{R}$, where we are interested in the joint distribution of:

$$ $\displaystyle\hat{\ell}=\frac{I\overline{o}}{r}\Tr\left(\bm{\rho}\bm{X}^{ \intercal}\bm{\tilde{O}}\bm{X}\right),$ (220) $\displaystyle\hat{\ell}_{;i}=\frac{I\overline{o}}{rN}\Tr\left(\bm{\rho}\bm{X}^ {\intercal}\left[\bm{X}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{X }^{\intercal},\bm{\tilde{O}}\right]\bm{X}\right),$ (221) $\displaystyle\begin{aligned} \hat{\ell}_{;i,j}=\frac{I\overline{o}}{rN^{2}}\Tr \Big(\bm{\rho}\bm{X}^{\intercal}\Big{[}&\bm{X}\left(\ket{2j}\bra{2j+1}-\ket{2j +1}\bra{2j}\right)\bm{X}^{\intercal},{\\ }\left[\bm{X}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{X}^{ \intercal},\bm{\tilde{O}}\right]\Big{]}\bm{X}\Big{missing}),\end{aligned}$ (222) $$

where now $\bm{X}$ is a $N\times\left(2p+1\right)$ random matrix with i.i.d. standard Gaussian entries over $\mathbb{R}$. As the commutator of symmetric matrices is antisymmetric, and taking note of the identity:

$$ $\Tr\left(A\left[B,C\right]\right)=\Tr\left(\left[C,A\right]B\right),$ (223) $$

we can rewrite this as:

$$ $\displaystyle\hat{\ell}$ $\displaystyle=\frac{I\overline{o}}{r}\Tr\left(\bm{\rho}\bm{X}^{\intercal}\bm{ \tilde{O}}\bm{X}\right),$ (224) $\displaystyle\hat{\ell}_{;i}$ $\displaystyle=\frac{2I\overline{o}}{rN}\Tr\left(\bm{\rho}\bm{X}^{\intercal} \left[\bm{X}\ket{2i}\bra{2i+1}\bm{X}^{\intercal},\bm{\tilde{O}}\right]\bm{X} \right),$ (225) $\displaystyle\hat{\ell}_{;i,j}$ $\displaystyle=\frac{2I\overline{o}}{rN^{2}}\Tr\left(\bm{\rho}\bm{X}^{\intercal }\left[\bm{X}\ket{2j}\bra{2j+1}\bm{X}^{\intercal},\left[\bm{X}\left(\ket{2i} \bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{X}^{\intercal},\bm{\tilde{O}}\right] \right]\bm{X}\right).$ (226) $$

Recall that

$$ $\bm{\tilde{O}}=\sum_{\mu=0}^{r-1}\ket{\mu}\bra{\mu}.$ (227) $$

Let $\bm{C}$ be the complement of $\bm{\tilde{O}}$, i.e.,

$$ $\bm{C}=\sum_{\mu=r}^{N-1}\ket{\mu}\bra{\mu}.$ (228) $$

In particular, $\bm{C}$ and $\bm{\tilde{O}}$ sum to the identity:

$$ $\bm{C}+\bm{\tilde{O}}=\bm{I}_{N}.$ (229) $$

Furthermore,

$$ $\displaystyle\bm{W}$ $\displaystyle\equiv\bm{X}^{\intercal}\bm{\tilde{O}}\bm{X},$ (230) $\displaystyle\bm{V}$ $\displaystyle\equiv\bm{X}^{\intercal}\bm{C}\bm{X},$ (231) $$

are each Wishart-distributed matrices; the former with $r$ degrees of freedom, and the latter with $N-r$. The independence of $\bm{W}$ and $\bm{V}$ can be checked from the associated Bartlett decompositions. We thus have that the distribution we are interested in is the joint distribution of:

$$ $\displaystyle\hat{\ell}=\frac{I\overline{o}}{r}\bra{0}\bm{W}\ket{0},$ (232) $\displaystyle\begin{aligned} \hat{\ell}_{;i}&=\frac{2I\overline{o}}{rN}\bra{0} \left(\bm{X}^{\intercal}\bm{X}\ket{2i}\bra{2i+1}\bm{W}-\bm{W}\ket{2i}\bra{2i+1 }\bm{X}^{\intercal}\bm{X}\right)\ket{0}\\ &=-\frac{2I\overline{o}}{rN}\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1} \bra{2i}\right)\bm{V}\ket{0},\end{aligned}$ (233) $$

and:

$$ $\displaystyle\hat{\ell}_{;i,j}=$ $\displaystyle\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\intercal}\left[\bm{X }\ket{2j}\bra{2j+1}\bm{X}^{\intercal},\left[\bm{X}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{X}^{\intercal},\bm{O}\right]\right]\bm{X}\ket{0}$ (234) $\displaystyle=$ $\displaystyle\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\intercal}\bm{X}\ket{ 2j}\bra{2j+1}\bm{X}^{\intercal}\bm{X}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2 i}\right)\bm{X}^{\intercal}\bm{O}\bm{X}\ket{0}$ $\displaystyle+\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\intercal}\bm{O}\bm{ X}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{X}^{\intercal}\bm{X} \ket{2j}\bra{2j+1}\bm{X}^{\intercal}\bm{X}\ket{0}$ $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\intercal}\bm{X} \left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{X}^{\intercal}\bm{O}\bm{ X}\ket{2j}\bra{2j+1}\bm{X}^{\intercal}\bm{X}\ket{0}$ $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{X}^{\intercal}\bm{X}\ket {2j}\bra{2j+1}\bm{X}^{\intercal}\bm{O}\bm{X}\left(\ket{2i}\bra{2i+1}-\ket{2i+1 }\bra{2i}\right)\bm{X}^{\intercal}\bm{X}\ket{0}$ $\displaystyle=$ $\displaystyle\frac{2I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \ket{2j}\bra{2j+1}\left(\bm{W}+\bm{V}\right)\left(\ket{2i}\bra{2i+1}-\ket{2i+1 }\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle+\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{W}\left(\ket{2i}\bra{2i+ 1}-\ket{2i+1}\bra{2i}\right)\left(\bm{W}+\bm{V}\right)\ket{2j}\bra{2j+1}\left( \bm{W}+\bm{V}\right)\ket{0}$ $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{W}\ket{2j}\bra{2j+1} \left(\bm{W}+\bm{V}\right)\ket{0}$ $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\bra{0}\left(\bm{W}+\bm{V}\right) \ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \left(\bm{W}+\bm{V}\right)\ket{0}.$ $$

We can simplify the expression for the second derivative by grouping terms by their order in $\bm{W}$. Note first that all terms cubic in matrix elements of $\bm{W}$ sum to zero. For the quadratic terms, note that for the outer two terms:

$$ $\displaystyle+\bra{0}\bm{V}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ (235) $\displaystyle+\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{V}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{V}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{V}\ket{0}$ $\displaystyle=$ $\displaystyle+\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{V}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{V}\ket{0},$ $$

and for the inner two terms:

$$ $\displaystyle+\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{V}\ket{2j}\bra{2j+1}\bm{W}\ket{0}$ (236) $\displaystyle+\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{W}\ket{2j}\bra{2j+1}\bm{V}\ket{0}$ $\displaystyle-\bra{0}\bm{V}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{W}\ket{2j}\bra{2j+1}\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{W}\ket{2j}\bra{2j+1}\bm{V}\ket{0}$ $\displaystyle=$ $\displaystyle+\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{V}\ket{2j}\bra{2j+1}\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{V}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{W}\ket{2j}\bra{2j+1}\bm{W}\ket{0}.$ $$

We can, in turn, combine these terms to obtain:

$$ $\displaystyle+\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{V}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ (237) $\displaystyle-\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{V}\ket{0}$ $\displaystyle+\bra{0}\bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{V}\ket{2j}\bra{2j+1}\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{V}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right) \bm{W}\ket{2j}\bra{2j+1}\bm{W}\ket{0}$ $\displaystyle=$ $\displaystyle+\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{V}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\ket{2j}\bra{2j+1}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{V}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\ket{2j+1}\bra{2j}\bm{V}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle+\bra{0}\bm{W}\ket{2j+1}\bra{2j}\bm{W}\left(\ket{2i}\bra{2i+1}- \ket{2i+1}\bra{2i}\right)\bm{V}\ket{0}$ $\displaystyle=$ $\displaystyle+\bra{0}\bm{W}\left(\ket{2j}\bra{2j+1}-\ket{2j+1}\bra{2j}\right) \bm{V}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{W}\ket{0}$ $\displaystyle-\bra{0}\bm{W}\left(\ket{2j}\bra{2j+1}-\ket{2j+1}\bra{2j}\right) \bm{W}\left(\ket{2i}\bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{V}\ket{0}.$ $$

We can then combine terms with those linear in $\bm{W}$ to see that:

$$ $\displaystyle\hat{\ell}_{;i,j}$ $\displaystyle=\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{W}\left(\ket{2i}\bra{2i+ 1}-\ket{2i+1}\bra{2i}\right)\bm{V}\left(\ket{2j}\bra{2j+1}-\ket{2j+1}\bra{2j} \right)\left(\bm{W}+\bm{V}\right)\ket{0}$ (238) $\displaystyle-\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{V}\left(\ket{2i}\bra{2i+ 1}-\ket{2i+1}\bra{2i}\right)\bm{W}\left(\ket{2j}\bra{2j+1}-\ket{2j+1}\bra{2j} \right)\left(\bm{W}+\bm{V}\right)\ket{0}.$ $$

We now use two lemmas—proved in Appendix M—that will simplify the form of our answer by allowing us to justify the dropping of terms that are subleading w.h.p. In particular, we use Lemma 43 followed by Lemma 41 just as we did in Eqs. (198), (199), and (200). By the same logic, we have that we can take up to an order

$$ $\alpha_{5}=\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}\left(N-r \right)^{\frac{1}{3}}\log\left(\frac{p^{2}N^{\frac{2}{3}}}{\mathcal{N}^{\frac{ 1}{3}}\left(N-r\right)^{\frac{1}{3}}}\right)}{N^{\frac{2}{3}}}\right)\leq \operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}\sigma_{o}^{\frac{2}{3}} \log\left(\frac{p^{2}N^{\frac{1}{3}}\overline{o}^{\frac{2}{3}}}{\mathcal{N}^{ \frac{1}{3}}\sigma_{o}^{\frac{2}{3}}}\right)}{N^{\frac{1}{3}}\overline{o}^{ \frac{2}{3}}}\right)$ (239) $$

error in Ky Fan metric:

$$ $\hat{\ell}_{;i,j}=-\frac{2I\overline{o}}{rN^{2}}\bra{0}\bm{V}\left(\ket{2i} \bra{2i+1}-\ket{2i+1}\bra{2i}\right)\bm{\tilde{W}}\left(\ket{2j}\bra{2j+1}- \ket{2j+1}\bra{2j}\right)\bm{W}\ket{0},$ (240) $$

where $\bm{\tilde{W}}$ is as $\bm{W}$, but with the first column of its Bartlett factor $\bm{L}_{W}$ removed.

At this point it is instructive to consider this joint distribution in terms of the marginal distributions of elements in the Bartlett decompositions of $\bm{W}$ and $\bm{V}$. Let $N_{W,i,j}$ be the i.i.d. standard normally distributed random variables over $\mathbb{R}$ in the off-diagonal elements of $\bm{L}_{W}$ (with the convention $i<j$), and similarly for $N_{V,i,j}$ and the Bartlett factor $\bm{L}_{V}$ of $\bm{V}$. Furthermore, let $\chi_{W,i}$ be the i.i.d. $\chi$-distributed random variables along the diagonal of $\bm{L}_{W}$, and similarly for $\chi_{V,i}$ and $\bm{L}_{V}$.

We first condition on $\hat{\ell}=z$. From Eq. (192) this is equivalent to conditioning on $\chi_{W,0}=\sqrt{\frac{r}{I\overline{o}}z}$. Conditioned on this event, the first derivative is distributed as:

$$ $\hat{\ell}_{;i\mid z}=\frac{2I\overline{o}}{rN}\sqrt{\frac{r}{I\overline{o}}z} \chi_{V,0}\left(N_{W,0,2i+1}N_{V,0,2i}-N_{W,0,2i}N_{V,0,2i+1}\right).$ (241) $$

Similarly, the second derivative conditioned on this event is distributed as:

$$ $\displaystyle\hat{\ell}_{;i,j\mid z}=$ $\displaystyle\frac{2I\overline{o}}{rN^{2}}\sqrt{\frac{r}{I\overline{o}}z}\chi_ {V,0}$ (242) $\displaystyle\times\left(N_{V,0,2i+1}\bra{2i}-N_{V,0,2i}\bra{2i+1}\right)\bm{ \tilde{W}}\left(\ket{2j}N_{W,0,2j+1}-\ket{2j+1}N_{W,0,2j}\right).$ $$

Note that $\bm{\tilde{W}}$ is independent from $N_{W,0,i}$ exactly due to the removal of the first column of the Bartlett factor of $\bm{W}$.

We will next simplify further using the fact that $\chi_{V,0}$ concentrates around its mean, in particular using the $\chi$-distribution tail bound (for $N-r>0$, some universal constant $C>0$, and bounded $t$):

$$ $\mathbb{P}\left[\frac{1}{\left(N-r\right)^{\frac{1}{4}}}\left\lvert\chi-\sqrt{ \beta\left(N-r\right)}\right\rvert\geq t\right]\leq 2\exp\left(-C\sqrt{N-r} \min\left(t,t^{2}\right)\right)$ (243) $$

which follows from Lemma 42, proved in Appendix M. Just as we accounted for $\alpha_{5}$, up to an order

$$ $\alpha_{6}=\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}}r^{\frac{1}{6} }\left(N-r\right)^{\frac{1}{12}}\log\left(\frac{p^{2}N^{\frac{2}{3}}}{\mathcal {N}^{\frac{1}{3}}r^{\frac{1}{6}}\left(N-r\right)^{\frac{1}{12}}}\right)}{N^{ \frac{2}{3}}}\right)\leq\operatorname{O}\left(\frac{\mathcal{N}^{\frac{1}{3}} \sigma_{o}^{\frac{1}{6}}\log\left(\frac{p^{2}N^{\frac{5}{12}}\overline{o}^{ \frac{1}{6}}}{\mathcal{N}^{\frac{1}{3}}\sigma_{o}^{\frac{1}{6}}}\right)}{N^{ \frac{5}{12}}\overline{o}^{\frac{1}{6}}}\right)$ (244) $$

error in Ky Fan metric,

$$ $\displaystyle\hat{\ell}_{;i\mid z}=\frac{2I\overline{o}}{N}\sqrt{\left(\frac{N }{r}-1\right)\frac{z}{I\overline{o}}}\left(N_{W,0,2i+1}N_{V,0,2i}-N_{W,0,2i}N_ {V,0,2i+1}\right),$ (245) $\displaystyle\begin{aligned} \hat{\ell}_{;i,j\mid z}=&\frac{2I\overline{o}}{N^ {2}}\sqrt{\left(\frac{N}{r}-1\right)\frac{z}{{I\overline{o}}}}\\ &\times\left(N_{V,0,2i+1}\bra{2i}-N_{V,0,2i}\bra{2i+1}\right)\bm{\tilde{W}} \left(\ket{2j}N_{W,0,2j+1}-\ket{2j+1}N_{W,0,2j}\right).\end{aligned}$ (246) $$

Using Lemma 25 we can simplify this further as:

$$ $\displaystyle\hat{\ell}_{;i\mid z}$ $\displaystyle=\frac{2I\sigma_{o}}{N}\sqrt{\frac{z}{I\overline{o}}}\left(N_{W,0 ,2i+1}N_{V,0,2i}-N_{W,0,2i}N_{V,0,2i+1}\right),$ (247) $\displaystyle\hat{\ell}_{;i,j\mid z}$ $\displaystyle=\frac{2I\sigma_{o}}{N^{2}}\sqrt{\frac{z}{{I\overline{o}}}}\left( N_{V,0,2i+1}\bra{2i}-N_{V,0,2i}\bra{2i+1}\right)\bm{\tilde{W}}\left(\ket{2j}N_ {W,0,2j+1}-\ket{2j+1}N_{W,0,2j}\right).$ (248) $$

We can simplify this even further by realizing that the joint distribution of $\left(N_{V,0,2i},N_{V,0,2i+1}\right)$ is invariant under orthogonal transformations, i.e., it is circularly symmetric. We can thus simultaneously transform $\left(N_{V,0,2i},N_{V,0,2i+1}\right)$ and conjugate $\bm{\tilde{W}}$ by orthogonal operators such that we need only consider the joint distribution of:

$$ $\displaystyle\hat{\ell}_{;i\mid z}$ $\displaystyle=\frac{2I\sigma_{o}}{N}\sqrt{\frac{z}{I\overline{o}}}\chi_{i}N_{V ,0,2i+1},$ (249) $\displaystyle\hat{\ell}_{;i,j\mid z}$ $\displaystyle=\frac{2I\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I\overline{o}}}\chi_{j} \left(N_{V,0,2i}\bra{2i}-N_{V,0,2i+1}\bra{2i+1}\right)\bm{\tilde{W}}\ket{2j},$ (250) $$

where here the $\chi_{i}$ are i.i.d. $\chi$-distributed random variables with $2$ degrees of freedom given by:

$$ $\chi_{i}\equiv\sqrt{N_{W,0,2i}^{2}+N_{W,0,2i+1}^{2}}.$ (251) $$

This proves Theorem 17 for sectors with $\mathbb{F}=\mathbb{R}$, with accumulated error in Lévy–Prokhorov distance:

$$ $\pi^{\prime}=\pi_{1}+\alpha_{3}+2\alpha_{4}+\alpha_{6}=\operatorname{O}\left( \frac{p\log\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}{\log\left(\mathcal{ N}^{-t}\epsilon^{-1}\right)}+\frac{p\log\left(t\right)}{\sqrt{t}}+\sqrt{\frac{ \mathcal{N}\log\left(N\right)}{N^{1.001}}}+\frac{\mathcal{N}^{\frac{1}{3}}\log \left(N\right)}{N^{\frac{5}{12}}}\right).$ (252) $$

We finally consider the second derivative conditioned on both $\hat{\ell}=z$ and all $\hat{\ell}_{;i\mid z}=0$ (for $z>0$). Note that the $\chi$ distribution density is zero at 0 0. In particular, this conditioning is equivalent to conditioning on $N_{V,0,2i+1}=0$, yielding:

$$ $\hat{\ell}_{;i,j\mid z,\bm{0}}=\frac{2I\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I \overline{o}}}N_{V,0,2i}\chi_{j}\bra{2i}\bm{\tilde{W}}\ket{2j}.$ (253) $$

As submatrices of Wishart matrices are also Wishart, we can more conveniently rewrite this by taking $\tilde{W}$ to be $p\times p$:

$$ $\hat{\ell}_{;i,j\mid z,\bm{0}}=\frac{2I\sigma_{o}}{N^{2}}\sqrt{\frac{z}{I \overline{o}}}\hat{N}_{i}\chi_{j}\bra{i}\bm{\tilde{W}}\ket{j},$ (254) $$

where the $\hat{N}_{i}$ are i.i.d. standard normal random variables. This proves Theorem 19 for sectors with $\mathbb{F}=\mathbb{R}$, with accumulated error in Lévy–Prokhorov distance:

$$ $\pi^{\prime\prime}=\pi_{1}+\alpha_{3}+2\alpha_{4}+\alpha_{5}+\alpha_{6}= \operatorname{O}\left(\frac{p^{2}\log\log\left(\mathcal{N}^{-t}\epsilon^{-1} \right)}{\log\left(\mathcal{N}^{-t}\epsilon^{-1}\right)}+\frac{p^{2}\log\left( t\right)}{\sqrt{t}}+\frac{\mathcal{N}^{\frac{1}{3}}\log\left(N\right)}{N^{ \frac{1}{3}}}\right).$ (255) $$

## Appendix E Open Questions

As our results are formal, there is room for violation of their conclusions in the settings where their assumptions do not hold. One example of this is the local minima distribution of quantum neural networks discussed in Appendix C.3. The results given there assume that the eigenvalue spectrum of the objective observable is concentrated as is typical of low-weight fermionic [] and local spin Hamiltonians []. However, the behavior of the local minima density differs when the spectrum is not concentrated, as is typical of observables drawn from the Gaussian unitary ensemble (GUE) and nonlocal spin systems. Observables from this latter class are known to be “quantumly easy” in the sense that phase estimation performed on the maximally mixed state efficiently prepares the ground state []. Based on this intuition these nonlocal systems may also yield local minima distributions for quantum neural networks more amenable to variational optimization even at shallow depth. If this is true, this may give a natural class of optimization problems which are easy to solve given access to a quantum computer.

Furthermore, we nowhere discuss the impact of noise: we assume all operations implemented by the quantum neural network are unitary, not general quantum channels. It is already known that noise can affect the presence of barren plateaus [] as well as local minima [, ] in variational loss landscapes. Heuristically the former can be understood as an additional channel mixing the variational ansatz over Hilbert space, and the latter as noise effectively limiting the number of parameters influencing the variational ansatz and thus always keeping the network in the underparameterized regime. We hope in the future to make this heuristic understanding rigorous.

## Appendix F Reduction of Spin Factor Sectors to Real Symmetric Sectors

We elaborate here on the claim made in Appendix B.1 that the $\alpha$ where $\mathcal{A}_{\alpha}\cong\mathcal{L}_{N_{\alpha}}$—i.e., the spin factor sectors—are conceptually equivalent to the case $\mathcal{A}_{\alpha}\cong\mathcal{H}^{N_{\alpha}-1}\left(\mathbb{R}\right)$. To see this, consider a parameterization of $T^{\alpha}\left(\bm{\theta}\right)$ in the spin factor case:

$$ $T\left(\bm{\theta}\right)=g_{0}^{\alpha\intercal}\prod_{i=1}^{p}g_{i}^{\alpha} \exp\left(\theta_{i}A_{i}^{\alpha}\right)g_{i}^{\alpha\intercal},$ (256) $$

where $g_{i}^{\alpha}\in\operatorname{SO}\left(N-1\right)$ and $A_{i}^{\alpha}\in\mathfrak{so}\left(N-1\right)$. As stated in Appendix B.1, $g_{i}^{\alpha}$ acts trivially on the first component in the defining representation. Because of this (assuming the loss is shifted by a constant to always be nonnegative), the associated $\ell^{\alpha}\left(\bm{\theta};\bm{\rho}\right)$ is just equal to the square root of an instance of the $\mathcal{H}^{N_{\alpha}-1}\left(\mathbb{R}\right)$ case with initial state:

$$ $\bm{\tilde{\rho}}^{\alpha}\equiv\bm{\rho}^{\alpha\intercal}\otimes\bm{\rho}^{\alpha}$ (257) $$

and objective observable:

$$ $\bm{\tilde{O}}^{\alpha}\equiv\bm{O}^{\alpha}\otimes\bm{O}^{\alpha\intercal}.$ (258) $$

## Appendix G Reduction of Previously Studied Ansatz Classes to Jordan Algebra-Supported Ansatzes

We here show that Jordan algebra-supported ansatzes (JASAs) are reduced to by the algebraically defined classes of variational ansatzes previously introduced in the literature, with a primary focus on the Lie algebra-supported ansatzes (LASAs) []. A reduction in the other direction—i.e., given a Jordan algebra-defined ansatz, define an equivalent LASA—is not possible. As a simple example, consider the case when the objective observable $\bm{O}$ is a real symmetric matrix and the variational ansatz is PT symmetric (i.e., real) but has no other symmetries. This fits our framework with Jordan algebra $\mathcal{A}\cong\mathcal{H}^{N}\left(\mathbb{R}\right)$ and automorphism group given by the action of conjugation by $\operatorname{SO}\left(N\right)$. However, $\mathfrak{so}\left(N\right)$ in the defining representation consists of antisymmetric matrices, so this is not a LASA. We also give a reduction from the variational matchgate circuits of Diaz *et al.* [] to our Jordan algebraic framework, and will later demonstrate that such a reduction to LASAs is not possible.

We first discuss how JASAs are reduced to by LASAs. The loss function of a LASA is of the form:

$$ $\ell\left(\bm{\theta};\rho\right)=\tau\left(\rho,T\left(\bm{\theta}\right)O \right),$ (259) $$

where $O$ belongs to some dynamical Lie algebra $\mathfrak{g}$, $T\left(\bm{\theta}\right)$ is the adjoint action of some element of the compact Lie group $G$ associated with $\mathfrak{g}$, and $\rho$ is arbitrary. As demonstrated in Fontana *et al.* [], only the projection of $\rho$ onto the $\mathfrak{g}$ contributes to the loss, so we consider when $\rho\in\mathfrak{g}$ WLOG.

$G$ is a compact Lie group. Thus, up to an Abelian sector that does not contribute to the loss (as $O\in\mathfrak{g}$), $G$ can be written as the direct sum of simple compact Lie groups. By the well-known classification of simple compact Lie groups these are either the classical $\mathfrak{so}\left(N\right)$, $\mathfrak{su}\left(N\right)$, $\mathfrak{sp}\left(N\right)$, or one of the exceptional Lie groups. The exceptional Lie groups are of fixed dimension—i.e., there is no sense of an asymptotic limit—so we consider here the classical cases only.

We begin with the case $\mathfrak{g}\cong\mathfrak{su}\left(N\right)$, with defining representation $\differential{\phi}$ over $\mathbb{C}$. It is easy to see that $\mathrm{i}\differential{\phi}\left(O\right)$ and $\mathrm{i}\differential{\phi}\left(\rho\right)$ are in the defining representation of the Jordan algebra $\mathcal{H}\left(\mathbb{C}\right)$, with identical trace form. Such a simple component is thus equivalent to a JASA with simple subalgebra isomorphic to $\mathcal{H}^{N}\left(\mathbb{C}\right)$.

We now consider when $\mathfrak{g}\cong\mathfrak{su}\left(N\right)$, with defining representation $\differential{\phi}$ over $\mathbb{H}$. As the loss is real-valued we have the expansion in this sector:

$$ $\displaystyle\ell\left(\bm{\theta};\rho\right)\propto$ $\displaystyle\Tr\left(\real\left\{\mathrm{i}\differential{\phi}\left(\rho \right)\right\}\differential{\phi}\left(U\left(\bm{\theta}\right)\right)\real \left\{\mathrm{i}\differential{\phi}\left(O\right)\right\}\differential{\phi} \left(U\left(\bm{\theta}\right)\right)^{\dagger}\right)$ (260) $\displaystyle+\Tr\left(\real\left\{\mathrm{j}\differential{\phi}\left(\rho \right)\right\}\differential{\phi}\left(U\left(\bm{\theta}\right)\right)\real \left\{\mathrm{j}\differential{\phi}\left(O\right)\right\}\differential{\phi} \left(U\left(\bm{\theta}\right)\right)^{\dagger}\right)$ $\displaystyle+\Tr\left(\real\left\{\mathrm{k}\differential{\phi}\left(\rho \right)\right\}\differential{\phi}\left(U\left(\bm{\theta}\right)\right)\real \left\{\mathrm{k}\differential{\phi}\left(O\right)\right\}\differential{\phi} \left(U\left(\bm{\theta}\right)\right)^{\dagger}\right).$ $$

Each of these real parts is Hermitian and can be written as elements of the defining representation of $\mathcal{H}^{N}\left(\mathbb{H}\right)$, reducing this simple component to that of a JASA.

We finally consider when $\mathfrak{g}\cong\mathfrak{so}\left(N\right)$. There are more subtleties compared with the other classical Lie algebras as, in the defining representation over $\mathbb{R}$, there is no simple relation between Hermitian and anti-Hermitian operators. To make the following concrete, we here consider the representation of $\mathfrak{so}\left(N\right)$ defined by Majorana operators $\gamma_{i}$, with generators:

$$ $M_{ij}=\frac{1}{2}\left[\gamma_{i},\gamma_{j}\right],$ (261) $$

i.e., polynomials quadratic in Majorana operators. These are the generators of the spin algebra $\mathfrak{spin}$ associated with the vector space spanned by the $\gamma_{i}$ and thus is isomorphic to $\mathfrak{so}\left(N\right)$. Furthermore, the gates generated by the $M_{ij}$ are just matchgates. The loss function variance of variational matchgate circuits was studied in detail in Diaz *et al.* []. The authors there also consider when $O\not\in\mathfrak{g}$, making this an example of a barren plateau result beyond the LASA formalism.

We now demonstrate how this setting fits into the Jordan algebra picture. Let $\mathcal{A}$ be the Jordan algebra of Majorana fermions, with Jordan multiplication $\circ$ given by half the anticommutator. As in the Lie algebra picture, the dimension of this algebra grows exponentially with $n$; similarly, the Jordan algebra generated by $k$-body ($2<k<n$) Majorana terms has dimension growing exponentially with $n$, as does the generated algebra when acting on these $k$-body terms via conjugation by a matchgate. However, though these operators are not closed as an algebra under matchgate conjugation, they *are* closed as a vector space. Furthermore, the trace form of a $k$-body term $\psi$ and a $k^{\prime}$-body term $\psi^{\prime}$ is always zero when $k\neq k^{\prime}$. This allows us to instead consider $O$ as a member of a different Jordan algebra $\mathcal{A}^{\prime}$, with the same underlying vector space as $\mathcal{A}$ and new Jordan multiplication operation $\bullet$ defined as:

$$ $\psi\bullet\psi^{\prime}=\begin{cases}1&\text{if }\psi=\psi^{\prime},\\ \psi&\text{if }\psi^{\prime}=1,\\ \psi^{\prime}&\text{if }\psi=1,\\ 0&\text{otherwise}.\end{cases}$ (262) $$

Note that this new multiplication operation still satisfies the Jordan identity:

$$ $\displaystyle\left(\psi\bullet\psi^{\prime}\right)\left(\psi\bullet\psi\right)$ $\displaystyle=\psi\bullet\psi^{\prime}$ (263) $\displaystyle=\psi\bullet\left(\psi^{\prime}\bullet 1\right)$ $\displaystyle=\psi\bullet\left(\psi^{\prime}\bullet\left(\psi\bullet\psi\right )\right).$ $$

It also preserves the canonical trace form, as:

$$ $\Tr\left(L_{\circ}\left(\psi\circ\psi^{\prime}\right)\right)=2^{n}\delta_{\psi ,\psi^{\prime}}=\Tr\left(L_{\bullet}\left(\psi\bullet\psi^{\prime}\right) \right).$ (264) $$

Here, $L_{\bullet}$ is the linear transformation associated with the Jordan multiplication $\bullet$:

$$ $L_{\bullet}\left(u\right)v=u\bullet v,$ (265) $$

and similarly for $L_{\circ}$. The loss function in this sector is thus a JASA under the Jordan algebra $\mathcal{A}^{\prime}$, and thus both general matchgate ansatzes or the LASA setting with $\mathfrak{g}\cong\mathfrak{so}\left(N\right)$ reduce to JASAs.

Interestingly, one can show that the matchgate setting *cannot* generally be described by a LASA while maintaining the trace form, even when the Lie bracket is redefined. To see this, we consider the adjoint endomorphism

$$ $\operatorname{ad}_{\left[\cdot,\cdot\right]}\left(u\right)v=\left[u,v\right]$ (266) $$

associated with the Lie bracket $\left[\cdot,\cdot\right]$. The canonical trace form for Lie algebras is the *Killing form*, given by:

$$ $K\left(u,v\right)=\Tr\left(\operatorname{ad}_{\left[\cdot,\cdot\right]}\left(u \right)\operatorname{ad}_{\left[\cdot,\cdot\right]}\left(v\right)\right).$ (267) $$

Note that (as $\operatorname{ad}_{\left[\cdot,\cdot\right]}\left(u\right)$ is antisymmetric):

$$ $K\left(u,u\right)=-\left\lVert\operatorname{ad}_{\left[\cdot,\cdot\right]} \left(u\right)\right\rVert_{\text{F}}^{2}.$ (268) $$

For $u$ a Majorana monomial under the usual Lie algebra of Majoranas, this is just (minus) the number of terms anticommuting with the monomial. For $u$ a degree-$1$ monomial this will grow exponentially in the number of Majoranas $n$. If instead one considered defining a new Lie bracket $\left[\cdot,\cdot\right]^{\prime}$ that is zero when mapping from $k$-degree Majorana monomials to those of different degree, $K\left(\cdot,\cdot\right)$ would always be zero on degree-$1$ Majorana monomials—in particular, the trace form is not preserved when modifying the Lie bracket.

## Appendix H Reduction of Low-Purity Inputs to Maximally Mixed Inputs

We elaborate here on the claim made in Appendix B.2 that if the input state $\bm{\rho}$ projected into a simple sector $\mathcal{A}_{\alpha}$ has sufficiently low purity, the loss contribution from this sector does not contribute in any meaningful way. This is true even when the loss is rescaled by some $\mathcal{N}\leq\operatorname{O}\left(N^{0.99}\right)$ as we consider elsewhere. Intuitively this is due to $\bm{\rho}^{\alpha}$ being close to maximally mixed and thus the contribution to the loss in this algebraic sector is effectively constant. We formally state this claim as the following lemma.

###### Lemma 29 (Low purity inputs are approximately trivial) .

Let $\mathcal{J},\ell$ be as in Theorem 13. For a $\bm{\rho}\in\mathcal{R}$ such that for some $\alpha$:

$$ $\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)\leq N_{\alpha}^{-0.999},$ (269) $$

we have for sufficiently large $\epsilon^{-1},t$ that:

$$ $\operatorname{Var}\left[\mathcal{N}\ell^{\alpha}\left(\bm{\theta};\bm{\rho} \right)\right]\leq\operatorname{O}\left(\frac{\mathcal{N}^{2}}{N_{\alpha}^{1.9 97}}\right)+\operatorname{O}\left(\pi\right)=\operatorname{o}\left(1\right)$ (270) $$

for all $\mathcal{N}\leq\operatorname{O}\left(N^{0.99}\right)$.

###### Proof.

Assume $\Tr\left(\bm{\rho}^{\alpha}\right)=1$ WLOG by otherwise absorbing a factor of $\Tr\left(\bm{\rho}^{\alpha}\right)$ into $\bm{O}^{\alpha}$. Consider the decomposition into eigenspaces:

$$ $\bm{\rho}^{\alpha}=\bm{\sigma}^{\alpha}+\bm{\tau}^{\alpha},$ (271) $$

where the nonzero eigenvalues of $\bm{\sigma}^{\alpha}$ are the largest $\left\lfloor N_{\alpha}^{0.999}\right\rfloor$ eigenvalues of $\bm{\rho}^{\alpha}$. Iterate this procedure

$$ $M_{\alpha}\equiv\left\lceil\frac{N_{\alpha}}{\left\lfloor N_{\alpha}^{0.999} \right\rfloor}\right\rceil$ (272) $$

total times to yield the decomposition:

$$ $\bm{\rho}^{\alpha}=\bm{\sigma}^{\alpha}+\sum_{i=1}^{M_{\alpha}-1}\bm{\tau}_{i} ^{\alpha}.$ (273) $$

The standard deviation of a sum of random variables is upper bounded by the sums of the standard deviations. Furthermore, from Eq. (273) we have that

$$ $\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)\geq\Tr\left(\left(\bm{ \sigma}^{\alpha}\right)^{2}\right).$ (274) $$

We can thus upper bound the standard deviation of the loss by applying Corollary 20 to the terms in Eq. (273):

$$ $\displaystyle\operatorname{Var}\left[\mathcal{N}\ell^{\alpha}\left(\bm{\theta} ;\bm{\rho}\right)\right]$ $\displaystyle\leq\mathcal{N}^{2}M_{\alpha}^{2}\frac{\Tr\left(\left(\bm{O}^{ \alpha}\right)^{2}\right)\Tr\left(\left(\bm{\sigma}^{\alpha}\right)^{2}\right) }{\dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)}+\operatorname{O}\left( \pi\right)$ (275) $\displaystyle\leq\mathcal{N}^{2}M_{\alpha}^{2}\frac{\Tr\left(\left(\bm{O}^{ \alpha}\right)^{2}\right)\Tr\left(\left(\bm{\rho}^{\alpha}\right)^{2}\right)}{ \dim_{\mathbb{R}}\left(\mathfrak{g}_{\alpha}\right)}+\operatorname{O}\left(\pi\right)$ $\displaystyle\leq\operatorname{O}\left(\frac{\mathcal{N}^{2}}{N^{1.997}}\right )+\operatorname{O}\left(\pi\right),$ $$

yielding the final result.
∎

###### Lemma 29 (Low purity inputs are approximately trivial) .

###### Proof.

## Appendix I ϵ italic-ϵ \epsilon italic_ϵ -Approximate 2 2 2 2 -Designs Suffice for the 𝒢 i subscript 𝒢 𝑖 \mathcal{G}_{i} caligraphic_G start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT

We here argue that one need only assume that the $\mathcal{G}_{i}$ form $\epsilon$-approximate $2$-designs—not large-$t$ $t$-designs—if one takes the overall normalization $\mathcal{N}$ to be at most $\operatorname{o}\left(N^{\frac{2}{3}}\right)$ and if $p$ grows sufficiently slowly with the $N_{\alpha}$. More concretely, instead of taking Assumption 15 we may take the following assumption:

###### Assumption 30 (Scaling of parameter space with ansatz moments) .

The number of trained parameters $p$ satisfies(^9^99As in Assumption 15, this can be weakened to a bound on $p$ if one is only interested in the gradient.):

$$ $p^{2}\leq\operatorname{o}\left(\min_{\alpha}\left(\frac{\log\left(\mathcal{N}^ {-1}N_{\alpha}^{\frac{2}{3}}\right)}{\log\log\left(\mathcal{N}^{-1}N_{\alpha}^ {\frac{2}{3}}\right)}\right)\right),$ (276) $$

where the $\mathcal{G}_{i}^{\alpha}$ are i.i.d. $\epsilon$-approximate $2$-designs over $\mathcal{T}_{\alpha}=\operatorname{Aut}_{1}\left(\mathcal{A}_{\alpha}\right)$, where

$$ $\epsilon\leq\operatorname{O}\left(N_{\alpha}^{-\frac{2}{3}}\right).$ (277) $$

To achieve this we will rely on tools from random matrix theory to bound the error of various mixed cumulants of our expressions under such a change. In particular, we will use the existence of so-called *limit distributions of all orders*.

###### Definition 31 (Limit distribution of all orders [ 71 ] ) .

A sequence of sets of $N\times N$ random matrices $\left\{\bm{A}_{i}\right\}_{i=1}^{d}$ has a *limit distribution of all orders* if there exist functions $\alpha_{\bm{k}}$ such that the cumulants $\kappa_{\bm{k}}$ obey as $N\to\infty$:

$$ $\displaystyle\lim_{N\to\infty}N^{\left\lVert\bm{k}\right\rVert_{1}-2}\kappa_{ \bm{k}}$ $\displaystyle\left(\Tr\left(p_{1}\left(\bm{A}_{1},\ldots,\bm{A}_{s}\right) \right),\ldots,\Tr\left(p_{r}\left(\bm{A}_{1},\ldots,\bm{A}_{s}\right)\right)\right)$ (278) $\displaystyle=\alpha_{\bm{k}}\left(p_{1}\left(\bm{A}_{1},\ldots,\bm{A}_{s} \right),\ldots,p_{r}\left(\bm{A}_{1},\ldots,\bm{A}_{s}\right)\right)$ $$

for all fixed polynomials $p_{i}$. Here, $r\equiv\left\lVert\bm{k}\right\rVert_{0}$.

Simple classes of distributions which satisfy this property include constant matrices of bounded operator norm, Gaussian random matrices, Wishart random matrices, and Haar random unitaries. We here will be interested in a weaker notion of the existence of a limit distribution of all orders, requiring only that higher-order moments vanish sufficiently quickly and not necessarily that they have well-defined limits.

###### Definition 32 (Weak limit distribution of all orders) .

A sequence of sets of $N\times N$ random matrices $\left\{\bm{A}_{i}\right\}_{i=1}^{d}$ has a *weak limit distribution of all orders* if the cumulants $\kappa_{\bm{k}}$ obey as $N\to\infty$:

$$ $\kappa_{\bm{k}}\left(\Tr\left(p_{1}\left(\bm{A}_{1},\ldots,\bm{A}_{s}\right) \right),\ldots,\Tr\left(p_{r}\left(\bm{A}_{1},\ldots,\bm{A}_{s}\right)\right) \right)=\operatorname{O}\left(N^{2-\left\lVert\bm{k}\right\rVert_{1}}\right)$ (279) $$

for all fixed polynomials $p_{i}$. Here, $r\equiv\left\lVert\bm{k}\right\rVert_{0}$.

Products of matrices with limit distributions of all orders over $\mathbb{F}=\mathbb{R},\mathbb{C},\mathbb{H}$ were considered in Mingo and Popa [], Collins *et al.* [], Redelmeier [] respectively, allowing us to prove the following lemma.

###### Lemma 33 (Existence of weak limit distributions of all orders) .

Let $L_{i}$, $i\in\left[d\right]$ be multilinear functions of the form:

$$ $L_{i}=\Tr\left(\bm{M}_{i}\bm{h}\bm{O}\bm{h}^{\dagger}\right),$ (280) $$

where $\bm{h}$ is Haar random and independent from the $\bm{M}_{i}$. Assume all $\bm{M}_{i}$ have weak limit distributions of all orders. For any cumulant $\kappa_{\bm{k}}\left(L_{i_{1}},\ldots,L_{i_{r}}\right)$ with $\left\lVert\bm{k}\right\rVert_{1}\geq 3$,

$$ $\kappa_{\bm{k}}\left(L_{i_{1}},\ldots,L_{i_{r}}\right)=\operatorname{O}\left(N ^{2-\left\lVert\bm{k}\right\rVert_{1}}\right).$ (281) $$

###### Proof.

Following:

- 1.
Lemma 5.12 of Redelmeier [] when $\mathbb{F}=\mathbb{R}$;
- 2.
Remark 4.8 of Collins *et al.* [] when $\mathbb{F}=\mathbb{C}$;
- 3.
and Corollary 3.2 of Redelmeier [] when $\mathbb{F}=\mathbb{H}$,

the product of a matrix from a unitarily-invariant ensemble with any other independent matrix with a weak limit distribution of all orders has a weak limit distribution of all orders. This implies that the $\bm{M}_{i}\bm{h}\bm{O}\bm{h}^{\dagger}$ have a weak limit distribution of all orders. The vanishing of higher-order cumulants as in Eq. (281) then follows from Definition 32.
∎

We thus have the following lemma.

###### Lemma 34 (Weak convergence to Haar random $\bm{g}_{i}$ ) .

Assume Assumption 30. Let $L_{i}$, $i\in\left[d\right]$ be uniformly bounded multilinear functions of the form:

$$ $L_{i}=\Tr\left(\bm{\rho}\bm{h}\bm{O}\bm{h}^{\dagger}\prod_{j\in\mathcal{I}_{i} }\bm{g}_{j}\bm{M}_{j}\bm{g}_{j}^{\dagger}\right),$ (282) $$

where $\mathcal{I}_{i}$ is some index set associated with $i$ of constant cardinality, $\bm{h}$ is Haar random, $\bm{g}_{j}\sim\mathcal{G}_{j}$, $\bm{\rho}$ and the $\bm{M}_{i}$ are fixed with bounded trace norm, and $d=\operatorname{o}\left(\frac{\log\left(\mathcal{N}^{-1}N^{\frac{2}{3}}\right)
}{\log\log\left(\mathcal{N}^{-1}N^{\frac{2}{3}}\right)}\right)$. Let $\hat{L}_{i}$ be the same multilinear functions, where now the $\bm{g}_{j}$ are Haar random. The joint distribution of $\mathcal{N}L_{i}$ differs from the joint distribution of $\mathcal{N}\hat{L}_{i}$ by an error at most

$$ $\pi_{2}=\operatorname{O}\left(\frac{d\log\left(\mathcal{N}^{-1}N^{\frac{2}{3}} \right)}{\left(\mathcal{N}^{-1}N^{\frac{2}{3}}\right)^{0.99}}\right)$ (283) $$

in Lévy–Prokhorov metric for any $\mathcal{N}\leq\operatorname{O}\left(N\right)$.

###### Proof.

The $\bm{g}_{j}\bm{M}_{j}\bm{g}_{j}^{\dagger}$ have a limit distribution of all orders due to the $\bm{M}_{j}$ being fixed. Thus, iteratively one can show the trace arguments of the $L_{i}$ have weak limit distributions of all orders by the results cited in the proof of Lemma 33. Incorporating the bounded trace norm of the $\bm{M}_{i}$ and the normalization $\mathcal{N}$ gives an error for the cumulants of order $k>2$:

$$ $\tilde{\epsilon}=\operatorname{O}\left(\mathcal{N}^{k}N^{1-k}\right)\leq \operatorname{O}\left(\left(\mathcal{N}N^{-\frac{2}{3}}\right)^{k}\right).$ (284) $$

This also bounds the $k=1,2$ cases whenever

$$ $\epsilon\leq\operatorname{O}\left(N^{-\frac{2}{3}}\right),$ (285) $$

which is true by Assumption 30. The result then follows from Corollary 40, where $\mu$ is subleading as in the proof of Lemma 26.
∎

###### Assumption 30 (Scaling of parameter space with ansatz moments) .

###### Definition 31 (Limit distribution of all orders [ 71 ] ) .

###### Definition 32 (Weak limit distribution of all orders) .

###### Lemma 33 (Existence of weak limit distributions of all orders) .

###### Proof.

###### Lemma 34 (Weak convergence to Haar random 𝒈 i subscript 𝒈 𝑖 \bm{g}_{i} bold_italic_g start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT ) .

###### Proof.

## Appendix J Equicontinuity of Probability Densities

In Appendix D we prove that the joint distribution of the loss function and its first two derivatives for a certain random class of variational quantum loss functions converges in distribution to a fairly simple expression involving Wishart-distributed random matrices. However, Corollaries 14 and 18 are stated in terms of the *probability densities* of the loss and first derivative, and weak convergence does not necessarily imply pointwise convergence in densities. However, it is known that this *is* the case when the sequence of densities are *equicontinuous* and bounded []. Whether or not this is true depends on the details of the distributions $\bm{\mathcal{G}},\bm{H}$. As an example, we here show this is true for the loss when $\bm{\mathcal{G}},\bm{H}$ are Haar random, under reasonable assumptions (which we conjecture are not necessary) on the spectrum of the objective observable $\bm{O}$ projected into any simple component.

###### Lemma 35 (Equicontinuity of loss density with Haar random ansatz) .

Consider a sequence of $\bm{O}$ as $N\to\infty$ with eigenvalues:

$$ $0=o_{1}\leq o_{2}\leq\ldots\leq o_{N-1}\leq o_{N}=1,$ (286) $$

with mean eigenvalue $\overline{o}$ and $\frac{\Tr\left(\bm{O}\right)^{2}}{\Tr\left(\bm{O}^{2}\right)N}=\frac{r}{N}=
\operatorname{\Theta}\left(1\right)$. For $\bm{w}\in\mathbb{R}^{N-2}$, define:

$$ $\displaystyle E_{\bm{w}}$ $\displaystyle\equiv\sqrt{\bm{\tilde{o}}\cdot\left(\bm{w}\odot\bm{w}\right)},$ (287) $\displaystyle R_{\bm{w}}$ $\displaystyle\equiv\sqrt{\bm{w}^{\intercal}\cdot\bm{w}},$ (288) $$

where $\bm{\tilde{o}}$ is the vector $\left(o_{2},\ldots,o_{N-1}\right)$ and $\odot$ denotes the Hadamard product. Let $D_{x}$ be the domain of $\bm{w}$ satisfying:

$$ $E_{\bm{w}}^{2}\leq\overline{o}+N^{-\frac{1}{2}}x\leq 1+E_{\bm{w}}^{2}-R_{\bm{w }}^{2}.$ (289) $$

Assume

$$ $p_{\sqrt{N}\ell}\left(x\right)=\frac{1}{\sqrt{N\left(\overline{o}+N^{-\frac{1} {2}}x\right)}S_{N-1}}\int_{D_{x}}\differential{\bm{w}}\left(1-\left(\overline{ o}+N^{-\frac{1}{2}}x\right)^{-1}E_{\bm{w}}^{2}\right)^{-\frac{1}{2}}$ (290) $$

is locally Lipschitz and bounded at any $x$ as $N\to\infty$, with Lipschitz constants independent from $N$. Then the density of $\sqrt{r}\ell$ is bounded by a constant independent of $N$ and is equicontinuous as a sequence in $N$.

###### Proof.

By assumption $r=\operatorname{\Theta}\left(N\right)$ so we consider the density of $\sqrt{N}\ell$ WLOG. Let $\left\{\ket{o_{i}}\right\}_{i=1}^{N}$ be the eigenvectors of $\bm{O}$ associated with the eigenvalues $o_{i}$. Let:

$$ $\ket{u}=\sum_{i=1}^{N}u_{i}\ket{o_{i}},$ (291) $$

and let:

$$ $S\left(\bm{u}\right)=\sum_{i=2}^{N}o_{i}\left\lvert u_{i}\right\rvert^{2}.$ (292) $$

Let $p_{\sqrt{N}\ell}\left(x\right)$ be the density of $\sqrt{N}\ell=\sqrt{N}\overline{o}+x$. We have that:

$$ $p_{\sqrt{N}\ell}\left(x\right)=S_{\beta N-1}^{-1}\int_{\left\lVert\bm{u}\right \rVert_{2}=1}\differential{\bm{u}}\delta\left(\sqrt{N}\left(S\left(\bm{u} \right)-\overline{o}\right)-x\right).$ (293) $$

Here, $S_{\beta N-1}$ is the surface area of the $\left(\beta N-1\right)$-sphere:

$$ $S_{\beta N-1}=\frac{2\uppi^{\frac{\beta N}{2}}}{\operatorname{\Gamma}\left( \frac{\beta N}{2}\right)}.$ (294) $$

We can integrate out the phases of the $u_{i}$ and note that the delta function has no dependence on $u_{1}$ since $o_{1}=0$. Therefore,

$$ $p_{\sqrt{N}\ell}\left(x\right)=S_{N-1}^{-1}2^{N}\int_{\begin{subarray}{c}\left \lVert\bm{\tilde{u}}\right\rVert_{2}\leq 1,\\ \bm{\tilde{u}}\geq\bm{0}\end{subarray}}\differential{\bm{\tilde{u}}}\delta \left(\sqrt{N}\left(S\left(\bm{\tilde{u}}\right)-\overline{o}\right)-x\right).$ (295) $$

Here, $\bm{\tilde{u}}\in\mathbb{R}_{\geq 0}^{N-1}$ has indices in $2,\ldots,N$ such that $\tilde{u}_{i}=\left\lvert u_{i}\right\rvert$. We will similarly use $\bm{\tilde{o}}$ to denote $\bm{o}$ with the $o_{1}=0$ component projected out.

We now proceed to show equicontinuity and uniform boundedness of $p_{f}\left(x\right)$ by showing that the derivative of $p_{f}\left(x\right)$ is bounded everywhere by some $N$-independent constant. By symmetry we restrict to the components of $v_{N}\geq 0$, introducing an overall factor of $2^{\beta}$. We also introduce new coordinates $w_{2},\ldots,w_{N}$, where $w_{i}=\tilde{u}_{i}$ for $2\leq i\leq N-1$ and $w_{N}=S\left(\bm{\tilde{u}}\right)$. The constraint $\tilde{u}_{N}^{2}\geq 0$ becomes in these coordinates:

$$ $\sum_{i=2}^{N-1}o_{i}w_{i}^{2}\leq w_{N}.$ (296) $$

Only the final row of the resulting Jacobian is nontrivial and is equal to:

$$ $\bm{J_{N}}=2\bm{\tilde{o}}^{\intercal}\odot\bm{\tilde{u}}^{\intercal},$ (297) $$

where $\odot$ denotes the Hadamard product. This gives a Jacobian determinant of (recalling that $o_{N}=1$):

$$ $\det\left(\bm{J}\right)=2\tilde{u}_{N}=2\sqrt{w_{N}-\sum_{i=2}^{N-1}o_{i}w_{i} ^{2}}.$ (298) $$

Putting everything together yields a density:

$$ $p_{\sqrt{N}\ell}\left(x\right)=S_{N-1}^{-1}2^{N-2}\int_{D}\differential{\bm{w} }\frac{\delta\left(\sqrt{N}\left(w_{N}-\overline{o}\right)-x\right)}{\sqrt{w_{ N}-\sum_{i=2}^{N-1}o_{i}w_{i}^{2}}},$ (299) $$

where $D$ is the domain of positive $w_{i}$ obeying the normalization constraint of Eq. (296) as well as:

$$ $w_{N}+\sum_{i=2}^{N-1}\left(1-o_{i}\right)w_{i}^{2}\leq 1.$ (300) $$

Defining:

$$ $\displaystyle E_{\bm{\tilde{w}}}^{2}$ $\displaystyle=\sum_{i=2}^{N-1}o_{i}\tilde{w}_{i}^{2},$ (301) $\displaystyle R_{\bm{\tilde{w}}}^{2}$ $\displaystyle=\sum_{i=2}^{N-1}\tilde{w}_{i}^{2},$ (302) $$

and rescaling $w_{N}$ we can rewrite this as:

$$ $p_{\sqrt{N}\ell}\left(x\right)=\frac{1}{\sqrt{N}S_{N-1}}\int_{B^{N-2}} \differential{\bm{\tilde{w}}}\int_{\sqrt{N}E_{\bm{\tilde{w}}}^{2}}^{\sqrt{N} \left(E_{\bm{\tilde{w}}}^{2}+1-R_{\bm{\tilde{w}}}^{2}\right)}\differential{w_{ N}}\frac{\delta\left(w_{N}-\sqrt{N}\overline{o}-x\right)}{\sqrt{N^{-\frac{1}{2 }}w_{N}-E_{\bm{\tilde{w}}}^{2}}},$ (303) $$

where $B^{N-2}$ is the unit $\left(N-2\right)$-ball. Integrating out the delta function then yields:

$$ $p_{\sqrt{N}\ell}\left(x\right)=\frac{1}{\sqrt{N\left(\overline{o}+N^{-\frac{1} {2}}x\right)}S_{N-1}}\int_{D_{x}}\differential{\bm{\tilde{w}}}\left(1-\left( \overline{o}+N^{-\frac{1}{2}}x\right)^{-1}E_{\bm{\tilde{w}}}^{2}\right)^{- \frac{1}{2}},$ (304) $$

where $D_{x}$ is the domain of $\bm{\tilde{w}}$ satisfying:

$$ $E_{\bm{\tilde{w}}}^{2}\leq\overline{o}+N^{-\frac{1}{2}}x\leq 1+E_{\bm{\tilde{w }}}^{2}-R_{\bm{\tilde{w}}}^{2}.$ (305) $$

By assumption this is locally Lipschitz and bounded as $N\to\infty$ with Lipschitz constants independent from $N$, and thus is equicontinuous and bounded.
∎

We now discuss the main assumption of Lemma 35 in more detail, particularly the local Lipschitz continuity of Eq. (290). Note that the defining equations of $D_{x}$ imply that all $w_{i}^{2}\leq 1$, i.e., the domain of integration is always of $\operatorname{O}\left(1\right)$ volume. This holds true even if some $o_{i}$ vanish with $N$; it is apparent that these terms do not contribute to the integrand at leading order $\bm{O}$, and only the $\operatorname{\Theta}\left(1\right)$ eigenvalues contribute. Because of this, we will perform an example calculation of the Lipschitz constants of Eq. (290) when all $o_{i}=\overline{o}=\frac{1}{2}$ for $i\neq 1,N$, though due to this observation similar results hold for all reasonable spectrums.

For such $\bm{O}$,

$$ $E_{\bm{\tilde{w}}}^{2}=\frac{1}{2}R_{\bm{\tilde{w}}}^{2},$ (306) $$

so

$$ $\displaystyle p_{\sqrt{N}\ell}\left(x\right)$ $\displaystyle=\frac{\sqrt{2}}{\sqrt{N\left(1+2N^{-\frac{1}{2}}x\right)}S_{N-1} }\int_{D_{x}}\differential{\bm{\tilde{w}}}\left(1-\left(1+2N^{-\frac{1}{2}}x \right)^{-1}R_{\bm{\tilde{w}}}^{2}\right)^{-\frac{1}{2}}$ (307) $\displaystyle=\frac{\sqrt{2}S_{N-3}}{\sqrt{N\left(1+2N^{-\frac{1}{2}}x\right)} S_{N-1}}\int_{0}^{\sqrt{1-2N^{-\frac{1}{2}}\left\lvert x\right\rvert}} \differential{r}r^{N-3}\left(1-\left(1+2N^{-\frac{1}{2}}x\right)^{-1}r^{2} \right)^{-\frac{1}{2}}$ $\displaystyle=\frac{S_{N-3}}{\sqrt{2N\left(1+2N^{-\frac{1}{2}}x\right)}S_{N-1} }\int_{0}^{1-2N^{-\frac{1}{2}}\left\lvert x\right\rvert}\differential{r}r^{ \frac{N}{2}-2}\left(1-\left(1+2N^{-\frac{1}{2}}x\right)^{-1}r\right)^{-\frac{1 }{2}}$ $\displaystyle=\frac{\left(1-2N^{-\frac{1}{2}}\left\lvert x\right\rvert\right)^ {\frac{N}{2}-1}S_{N-3}}{\sqrt{2N\left(1+2N^{-\frac{1}{2}}x\right)}S_{N-1}}\int _{0}^{1}\differential{r}r^{\frac{N}{2}-2}\left(1-\frac{1-2N^{-\frac{1}{2}} \left\lvert x\right\rvert}{1+2N^{-\frac{1}{2}}x}r\right)^{-\frac{1}{2}}$ $\displaystyle=\frac{\left(1-2N^{-\frac{1}{2}}\left\lvert x\right\rvert\right)^ {\frac{N}{2}-1}\operatorname{\Gamma}\left(\frac{N}{2}\right)}{\uppi\sqrt{2N \left(1+2N^{-\frac{1}{2}}x\right)}\operatorname{\Gamma}\left(\frac{N}{2}-1 \right)}\int_{0}^{1}\differential{r}r^{\frac{N}{2}-2}\left(1-\frac{1-2N^{- \frac{1}{2}}\left\lvert x\right\rvert}{1+2N^{-\frac{1}{2}}x}r\right)^{-\frac{1 }{2}}.$ $$

This integral is just the integral definition of the hypergeometric function $\operatorname{\prescript{}{2}{F}_{1}}$, yielding:

$$ $p_{\sqrt{N}\ell}\left(x\right)=\frac{\left(1-2N^{-\frac{1}{2}}\left\lvert x \right\rvert\right)^{\frac{N}{2}-1}}{\uppi\sqrt{2N\left(1+2N^{-\frac{1}{2}}x \right)}}\operatorname{\prescript{}{2}{F}_{1}}\left(\frac{1}{2},\frac{N}{2}-1; \frac{N}{2};\frac{1-2N^{-\frac{1}{2}}\left\lvert x\right\rvert}{1+2N^{-\frac{1 }{2}}x}\right).$ (308) $$

We now consider the large-$N$ asymptotics of the derivative of this expression. By Gauss’s hypergeometric theorem, for $z=1+\operatorname{O}\left(N^{-\frac{1}{2}}\right)$,

$$ $\operatorname{\prescript{}{2}{F}_{1}}\left(\frac{1}{2},\frac{N}{2}-1;\frac{N}{ 2};z\right)=\left(1+\operatorname{O}\left(N^{-\frac{1}{2}}\right)\right)\frac{ \operatorname{\Gamma}\left(\frac{N}{2}\right)\operatorname{\Gamma}\left(\frac{ 1}{2}\right)}{\operatorname{\Gamma}\left(\frac{N-1}{2}\right)\operatorname{ \Gamma}\left(1\right)}=\operatorname{O}\left(\sqrt{N}\right).$ (309) $$

Similarly, we have by the derivative rule for the hypergeometric function that:

$$ $\operatorname{\prescript{}{2}{F}_{1}}^{\prime}\left(\frac{1}{2},\frac{N}{2}-1; \frac{N}{2};z\right)=\frac{1}{2}\left(1-2N^{-1}\right)\prescript{}{2}{F}_{1} \left(\frac{3}{2},\frac{N}{2};\frac{N}{2}+1;z\right)=\operatorname{O}\left(N^{ \frac{3}{2}}\right).$ (310) $$

Taking into account the $N^{-\frac{1}{2}}$ factors scaling $x$ thus yields (where defined):

$$ $p_{\sqrt{N}\ell}^{\prime}\left(x\right)=\operatorname{O}\left(1\right).$ (311) $$

In particular, $p_{\sqrt{N}\ell}\left(x\right)$ is locally Lipschitz with Lipschitz constants independent from $N$. These $\bm{O}$ thus satisfy the assumptions of Lemma 35, as claimed.

###### Lemma 35 (Equicontinuity of loss density with Haar random ansatz) .

###### Proof.

## Appendix K Bounding Large Deviations of the Hessian Determinant

In calculating the density of local minima in Appendix C.3 we require analyzing the asymptotics of:

$$ $\mathbb{E}\left[\det\left(\bm{D}_{z}\right)\bm{1}\left\{\bm{D}_{z}\succeq\bm{0 }\right\}\right],$ (312) $$

where $\bm{D}_{z}$ is as in Eq. (128), i.e., it is a Wishart-distributed random matrix normalized by its degrees of freedom parameter. As the determinant is exponentially sensitive to its argument, exponentially unlikely deviations in the convergence of the spectrum of $\bm{D}_{z}$ to its asymptotic value can, in principle, cause this expectation to not converge to the determinant of the (almost sure) asymptotic spectral measure of $\bm{D}_{z}$. The same is true of the minimum eigenvalue of $\bm{D}_{z}$. We here show that these deviations do not asymptotically contribute to Eq. (312). We will here be brief as this is not the main focus of our work, and instead refer the interested reader to Anschuetz [] for a more detailed account of very similar arguments. In the following we consider only the case when $\gamma\leq 1$ as otherwise $\bm{D}_{z}$ is never full-rank and Eq. (312) is identically zero.

We begin with the empirical spectral measure of $\bm{D}_{z}$. The empirical spectral measures of $\bm{W}$ satisfies a large deviations principle at a speed $p^{2}$ with good rate function maximized by the Marčenko–Pastur distribution $\differential{\mu_{\text{MP}}^{\gamma}\left(\lambda\right)}$ []. As the determinant is only sensitive at a speed $p$, i.e.,

$$ $\det\left(\bm{D}_{z}\right)=\exp\left(p\int\differential{\mu_{\text{MP}}^{ \gamma}\left(\lambda\right)}\ln\left(\lambda\right)\right),$ (313) $$

by Varadhan’s lemma [] the expected determinant converges to the determinant of our asymptotic expression for $\bm{D}_{z}$.

We now consider the smallest eigenvalue of $\bm{D}_{z}$. For fluctuations below its asymptotic value this satisfies a large deviations principle with good rate function at a speed $p$, where this rate function is maximized at the infimum of the support of the Marčenko–Pastur distribution; for fluctuations above its asymptotic value it satisfies a large deviations principle at a speed $p^{2}$ []. By Varadhan’s lemma, then, only fluctuations below the asymptotic value potentially contribute, but as these fluctuations only decrease the value of the argument of Eq. (312) they do not contribute asymptotically []. These together imply that Eq. (312) converges as claimed in Appendix C.3.

## Appendix L Lemmas on Convergence in Distribution

We here prove a few helper lemmas that will allow us to obtain quantitative error bounds on the rate of convergence of two sequences of distributions given bounds on the differences of their moments or cumulants. We will achieve this by bounding the *Lévy–Prokhorov distance* between the two distributions, which metricizes weak convergence. We first restate a result of Berkes and Philipp [] which bounds this distance, with a slight modification due to our use of the infinity norm rather than the Euclidean norm (see Appendix A.5).

###### Theorem 36 (Slightly modified version of Lemma 2.2, Berkes and Philipp [ 77 ] ) .

Let $p\left(\bm{x}\right),q\left(\bm{x}\right)$ be two distributions on $\mathbb{R}^{d}$ with characteristic functions $\phi\left(\bm{u}\right),\gamma\left(\bm{u}\right)$, respectively. Assume there exists a $C$ such that $\int_{\left\lVert\bm{x}\right\rVert_{\infty}\geq C}p\left(\bm{x}\right)\leq\mu$. There exists a universal constant $K$ such that for all $T\geq\max\left(2C,Kd\right)$, the Lévy–Prokhorov distance between $p$ and $q$ is bounded by:

$$ $\operatorname{\pi}\left(p,q\right)\leq\left(\frac{T}{\uppi}\right)^{d}\int_{ \left\lVert\bm{u}\right\rVert_{\infty}\leq T}\left\lvert\phi\left(\bm{u}\right )-\gamma\left(\bm{u}\right)\right\rvert\differential{\bm{u}}+\frac{16\ln\left( T\right)}{T}+\mu.$ (314) $$

Motivated by this, we prove a general bound on the error of the characteristic functions given errors in the moments.

###### Lemma 37 (Bound on characteristic functions from moments) .

Let $p\left(\bm{x}\right),q\left(\bm{x}\right)$ be two probability densities on $\mathbb{R}^{d}$ with characteristic functions $\phi\left(\bm{u}\right),\gamma\left(\bm{u}\right)$. Assume each moment of order $k\leq t$ of $p$ differs from that of $q$ by an additive error at most $\epsilon>0$, and assume all moments of order $k>t$ are bounded by $\left(C\sqrt{k}\right)^{k}$ for some $k$-independent $C\geq 0$. Then, for all $\bm{u}$ such that $\left\lVert\bm{u}\right\rVert_{1}\leq\frac{\sqrt{t+1}}{2eC}$,

$$ $\left\lvert\phi\left(\bm{u}\right)-\gamma\left(\bm{u}\right)\right\rvert\leq \epsilon\exp\left(\left\lVert\bm{u}\right\rVert_{1}\right)+\exp_{2}\left(1-t \right).$ (315) $$

###### Proof.

We have by the definition of the characteristic function and the triangle inequality that:

$$ $\left\lvert\phi\left(\bm{u}\right)-\gamma\left(\bm{u}\right)\right\rvert\leq \epsilon\sum_{\bm{k}\neq\bm{0}\in\mathbb{N}_{0}^{d}}\prod_{j=1}^{d}\frac{\left \lvert\left(\mathrm{i}u_{j}\right)^{k_{j}}\right\rvert}{k_{j}!}+2\sum_{\left \lVert\bm{k}\right\rVert_{1}\geq t+1}C^{\left\lVert\bm{k}\right\rVert_{1}} \left\lVert\bm{k}\right\rVert_{1}^{\frac{\left\lVert\bm{k}\right\rVert_{1}}{2} }\prod_{j=1}^{d}\frac{\left\lvert\left(\mathrm{i}u_{j}\right)^{k_{j}}\right \rvert}{k_{j}!}.$ (316) $$

The first term has the upper bound:

$$ $\epsilon\sum_{\bm{k}\neq\bm{0}\in\mathbb{N}_{0}^{d}}\prod_{j=1}^{d}\frac{\left \lvert\left(\mathrm{i}u_{j}\right)^{k_{j}}\right\rvert}{k_{j}!}\leq\epsilon \exp\left(\left\lVert\bm{u}\right\rVert_{1}\right).$ (317) $$

For the second term, we have the upper bound (for $\left\lVert\bm{u}\right\rVert_{1}<\frac{\sqrt{t+1}}{\mathrm{e}C}$):

$$ $\displaystyle\sum_{\left\lVert\bm{k}\right\rVert_{1}\geq t+1}C^{\left\lVert\bm {k}\right\rVert_{1}}\left\lVert\bm{k}\right\rVert_{1}^{\frac{\left\lVert\bm{k} \right\rVert_{1}}{2}}\prod_{j=1}^{d}\frac{\left\lvert u_{j}\right\rvert^{k_{j} }}{k_{j}!}$ $\displaystyle\leq\sum_{k\geq t+1}\frac{\left\lVert\sqrt{\mathrm{e}}C\bm{u} \right\rVert_{1}^{k}}{\sqrt{k!}}$ (318) $\displaystyle\leq\sum_{k\geq t+1}\left\lVert\frac{\mathrm{e}C\bm{u}}{\sqrt{t+1 }}\right\rVert_{1}^{k}$ $\displaystyle=\frac{\left(\frac{\mathrm{e}C}{\sqrt{t+1}}\left\lVert\bm{u} \right\rVert_{1}\right)^{t+1}}{1-\frac{\mathrm{e}C}{\sqrt{t+1}}\left\lVert\bm{ u}\right\rVert_{1}}.$ $$

When $\left\lVert\bm{u}\right\rVert_{1}\leq\frac{\sqrt{t+1}}{2\mathrm{e}C}$ this gives the bound:

$$ $2\sum_{\left\lVert\bm{k}\right\rVert_{1}\geq t+1}C^{\left\lVert\bm{k}\right \rVert_{1}}\left\lVert\bm{k}\right\rVert_{1}^{\frac{\left\lVert\bm{k}\right \rVert_{1}}{2}}\prod_{j=1}^{d}\frac{\left\lvert\left(\mathrm{i}u_{j}\right)^{k _{j}}\right\rvert}{k_{j}!}\leq 4\left(\frac{\mathrm{e}C}{\sqrt{t+1}}\left \lVert\bm{u}\right\rVert_{1}\right)^{t+1}\leq\exp_{2}\left(1-t\right).$ (319) $$

Combining these bounds yields the final result.
∎

This characteristic function bound combined with Theorem 36 gives us the following corollary.

###### Corollary 38 (Bound on Lévy–Prokhorov metric from moments) .

Let $p_{N}\left(\bm{x}\right),q_{N}\left(\bm{x}\right)$ be two sequences of distributions on $\mathbb{R}^{d}$ with characteristic functions $\phi_{N}\left(\bm{u}\right),\gamma_{N}\left(\bm{u}\right)$, respectively. Assume each moment of order $k\leq t$ of $p_{N}$ differs from that of $q_{N}$ by an ($N$-dependent) additive error at most $\epsilon>0$, and assume all moments of order $k>t$ (for $t$ $N$-dependent) are bounded by $\left(C\sqrt{k}\right)^{k}$ for some $\left(k,N\right)$-independent constant $C$. Assume as well that $\int_{\left\lVert\bm{x}\right\rVert_{\infty}\geq\frac{1}{2}\min\left(\frac{0.9
9}{d}\ln\left(\epsilon^{-1}\right),\frac{\sqrt{t+1}}{2\mathrm{e}Cd}\right)}p
\left(\bm{x}\right)\leq\mu$ for some ($N$-dependent) $\mu$. The Lévy–Prokhorov distance between $p_{N},q_{N}$ is bounded by:

$$ $\operatorname{\pi}\left(p_{N},q_{N}\right)=\operatorname{O}\left(\frac{d\log \log\left(\epsilon^{-1}\right)}{\log\left(\epsilon^{-1}\right)}+\frac{d\log \left(t\right)}{\sqrt{t}}+\mu\right).$ (320) $$

###### Proof.

Let $T=\min\left(\frac{0.99}{d}\ln\left(\epsilon^{-1}\right),\frac{\sqrt{t+1}}{2
\mathrm{e}Cd}\right)$. From Theorem 36,

$$ $\operatorname{\pi}\left(p_{N},q_{N}\right)\leq\operatorname{\tilde{O}}\left(T^ {2d}\right)\sup_{\left\lVert\bm{u}\right\rVert_{\infty}\leq T}\left\lvert\phi_ {N}\left(\bm{u}\right)-\gamma_{N}\left(\bm{u}\right)\right\rvert+\operatorname {O}\left(\frac{d\log\left(T\right)}{T}\right)+\operatorname{O}\left(\mu\right).$ (321) $$

From Lemma 37,

$$ $\sup_{\left\lVert\bm{u}\right\rVert_{\infty}\leq T}\left\lvert\phi_{N}\left( \bm{u}\right)-\gamma_{N}\left(\bm{u}\right)\right\rvert\leq\epsilon^{0.01}+ \exp_{2}\left(1-t\right).$ (322) $$

The convergence rate given in Eq. (320) is trivial when $d=\operatorname{\Omega}\left(\min\left(\frac{\log\left(\epsilon^{-1}\right)}{
\log\log\left(\epsilon^{-1}\right)},\frac{\sqrt{t}}{\log\left(t\right)}\right)\right)$ as $\epsilon^{-1},t,N\to\infty$. When instead $d=\operatorname{o}\left(\min\left(\frac{\log\left(\epsilon^{-1}\right)}{\log
\log\left(\epsilon^{-1}\right)},\frac{\sqrt{t}}{\log\left(t\right)}\right)\right)$,

$$ $T^{2d}=\exp\left(\operatorname{o}\left(\min\left(\log\left(\epsilon^{-1}\right ),\sqrt{t}\right)\right)\right).$ (323) $$

The second term in Eq. (321) thus dominates in this setting, implying the final bound.
∎

We also prove a bound on the error of characteristic functions given bounds on the cumulants.

###### Lemma 39 (Bound on characteristic functions from cumulants) .

Let $p\left(\bm{x}\right),q\left(\bm{x}\right)$ be two probability densities on $\mathbb{R}^{d}$ with characteristic functions $\phi\left(\bm{u}\right),\gamma\left(\bm{u}\right)$. Assume each cumulant of order $k$ of $p$ differs from that of $q$ by an additive error of at most $\epsilon^{k}>0$. Then

$$ $\left\lvert\phi\left(\bm{u}\right)-\gamma\left(\bm{u}\right)\right\rvert\leq \max\left(\exp\left(\exp\left(\epsilon\left\lVert\bm{u}\right\rVert_{1}\right) -1\right)-1,1-\exp\left(1-\exp\left(\epsilon\left\lVert\bm{u}\right\rVert_{1} \right)\right)\right).$ (324) $$

###### Proof.

We have that:

$$ $\displaystyle\left\lvert\phi\left(\bm{u}\right)-\gamma\left(\bm{u}\right)\right\rvert$ $\displaystyle\leq\left\lvert\phi\left(\bm{u}\right)\right\rvert\left\lvert 1- \exp\left(\ln\left(\gamma\left(\bm{u}\right)\right)-\ln\left(\phi\left(\bm{u} \right)\right)\right)\right\rvert$ (325) $\displaystyle=\left\lvert 1-\exp\left(\ln\left(\gamma\left(\bm{u}\right)\right )-\ln\left(\phi\left(\bm{u}\right)\right)\right)\right\rvert.$ $$

By the definition of the cumulant generating function,

$$ $\left\lvert\ln\left(\gamma\left(\bm{u}\right)\right)-\ln\left(\phi\left(\bm{u} \right)\right)\right\rvert\leq\sum_{\left\lVert\bm{k}\right\rVert_{1}>0}\prod_ {j=1}^{d}\frac{\left\lvert\left(\mathrm{i}\epsilon u_{j}\right)^{k_{j}}\right \rvert}{k_{j}!}=\exp\left(\epsilon\left\lVert\bm{u}\right\rVert_{1}\right)-1.$ (326) $$

Taking into account the absolute values yields the final result.
∎

This characteristic function bound combined with Theorem 36 gives us the following corollary.

###### Corollary 40 (Bound on Lévy–Prokhorov metric from cumulants) .

Let $p_{N}\left(\bm{x}\right),q_{N}\left(\bm{x}\right)$ be two sequences of distributions on $\mathbb{R}^{d}$ with characteristic functions $\phi_{N}\left(\bm{u}\right),\gamma_{N}\left(\bm{u}\right)$, respectively. Assume each cumulant of order $k$ of $p_{N}$ differs from that of $q_{N}$ by an additive error of at most $\epsilon^{k}>0$. Assume as well that $\int_{\left\lVert\bm{x}\right\rVert_{\infty}\geq\frac{1}{2d}\epsilon^{-0.99}}p
\left(\bm{x}\right)\leq\mu$ for some ($N$-dependent) $\mu$. Finally, assume that $d=\operatorname{o}\left(\frac{\log\left(\epsilon^{-1}\right)}{\log\log\left(
\epsilon^{-1}\right)}\right)$ as $N\to\infty$. The Lévy–Prokhorov distance between $p_{N},q_{N}$ is bounded by:

$$ $\operatorname{\pi}\left(p_{N},q_{N}\right)=\operatorname{O}\left(\frac{d\log \left(\epsilon^{-1}\right)}{\epsilon^{-0.99}}+\mu\right).$ (327) $$

###### Proof.

Let $T=\frac{1}{d}\epsilon^{-0.99}$. From Theorem 36,

$$ $\operatorname{\pi}\left(p_{N},q_{N}\right)\leq\operatorname{\tilde{O}}\left(T^ {2d}\right)\sup_{\left\lVert\bm{u}\right\rVert_{\infty}\leq T}\left\lvert\phi_ {N}\left(\bm{u}\right)-\gamma_{N}\left(\bm{u}\right)\right\rvert+\operatorname {O}\left(\frac{d\log\left(T\right)}{T}\right)+\operatorname{O}\left(\mu\right).$ (328) $$

From Lemma 39,

$$ $\sup_{\left\lVert\bm{u}\right\rVert_{\infty}\leq T}\left\lvert\phi_{N}\left( \bm{u}\right)-\gamma_{N}\left(\bm{u}\right)\right\rvert\leq\operatorname{ \Theta}\left(\epsilon^{0.01}\right).$ (329) $$

Similarly,

$$ $T^{2d}=\exp\left(\operatorname{o}\left(\log\left(\epsilon^{-1}\right)\right) \right).$ (330) $$

The second term in Eq. (328) thus dominates, implying the given convergence rate.
∎

###### Theorem 36 (Slightly modified version of Lemma 2.2, Berkes and Philipp [ 77 ] ) .

###### Lemma 37 (Bound on characteristic functions from moments) .

###### Proof.

###### Corollary 38 (Bound on Lévy–Prokhorov metric from moments) .

###### Proof.

###### Lemma 39 (Bound on characteristic functions from cumulants) .

###### Proof.

###### Corollary 40 (Bound on Lévy–Prokhorov metric from cumulants) .

###### Proof.

## Appendix M Tail Bound Lemmas

We here prove some helper lemmas giving tail bounds for the marginal entries of $\beta$-Wishart random variables. We will particularly leverage the Bartlett decomposition of $\beta$-Wishart matrices (see Appendix A.4). In the following we will use $\ket{\cdot}$ to denote vectors in the vector space on which the defining representation of $\mathcal{H}^{p+1}\left(\mathbb{F}\right)$ acts, with basis $\ket{0}\cup\left\{\ket{i}\right\}_{i=1}^{p}$.

We first prove that $\beta$-Wishart matrices are robust to certain low-rank perturbations of their Bartlett decomposition. In particular, we show that removing a single column from the Bartlett factor $\bm{L}$ of a $\beta$-Wishart matrix with many degrees of freedom—effectively reducing the degrees of freedom parameter of the matrix by one—has little effect on the joint distribution of entries of the matrix. Intuitively, this allows us to argue in Appendix D.3 that such a Wishart matrix is approximately independent from these entries.

###### Lemma 41 (Robustness of $\beta$ -Wishart Bartlett decompositions) .

Let $\bm{W}$ be a $\left(p+1\right)\times\left(p+1\right)$ $\beta$-Wishart matrix with $r$ degrees of freedom and identity scale matrix. Let $\bm{L}\bm{L}^{\dagger}$ be the Bartlett decomposition of $\bm{W}$. Let $\bm{\tilde{L}}$ be $\bm{L}$ with the first column removed, and define $\bm{\tilde{W}}\equiv\bm{\tilde{L}}\bm{\tilde{L}}^{\dagger}$. For every $t\geq 0$,

$$ $\mathbb{P}\left[r^{-\frac{1}{2}}\max_{i,j\in\left[1,\ldots,p\right]}\left \lvert\bra{i}\bm{W}\ket{j}-\bra{i}\bm{\tilde{W}}\ket{j}\right\rvert\geq t^{2}+ \sqrt{2}\beta^{-\frac{1}{2}}r^{-\frac{1}{4}}t+r^{-\frac{1}{2}}\right]\leq p \exp\left(-\frac{\sqrt{r}}{2}t^{2}\right).$ (331) $$

In particular, the argument converges to zero under the Ky Fan metric as $r\to\infty$ at a rate $\operatorname{\Omega}\left(\frac{\sqrt{r}}{\log\left(pr\right)}\right)$.

###### Proof.

Let $N_{0,i}$ be the $\left(i,0\right)$ entry of $\bm{L}$. Note that the $N_{0,i}$ are i.i.d. The lemma statement is then equivalent to showing that

$$ $\mathbb{P}\left[\max_{i,j\in\left[1,\ldots,p\right]}\left\lvert N_{0,i}N_{0,j} ^{\ast}\right\rvert^{2}\geq\left(\sqrt{r}t^{2}+\sqrt{2}\beta^{-\frac{1}{2}}r^{ \frac{1}{4}}t+1\right)^{2}\right]\leq p\exp\left(-\frac{\sqrt{r}}{2}t^{2} \right).$ (332) $$

This in turn is implied by:

$$ $\mathbb{P}\left[\beta\max_{i\in\left[1,\ldots,p\right]}\left\lvert N_{0,i} \right\rvert^{2}\geq\beta\sqrt{r}t^{2}+\sqrt{2\beta}r^{\frac{1}{4}}t+\beta \right]\leq p\exp\left(-\frac{\sqrt{r}}{2}t^{2}\right).$ (333) $$

$\beta\left\lvert N_{0,i}\right\rvert^{2}$ is $\chi^{2}$-distributed with $\beta$ degrees of freedom. The union bound along with standard $\chi^{2}$ tail bounds (see, e.g., Lemma 1 of Laurent and Massart []) thus immediately imply Eq. (333) and therefore also the final result.
∎

We now prove a tail bound for a $\chi$-distributed random variable with $D$ degrees of freedom.

###### Lemma 42 (Tail bounds for $\chi$ -distributed random variables) .

Let $\chi$ be $\chi$-distributed with $D$ degrees of freedom. For every $0\leq t\leq\frac{3}{8}D^{\frac{1}{4}}$,

$$ $\mathbb{P}\left[D^{-\frac{1}{4}}\left\lvert\chi-\sqrt{D}\right\rvert\geq\frac{ 4}{3}t+D^{-\frac{1}{4}}t^{2}\right]\leq 2\exp\left(-\sqrt{D}t^{2}\right).$ (334) $$

In particular, the argument converges to zero under the Ky Fan metric as $D\to\infty$ at a rate $\Omega\left(\frac{\sqrt[4]{D}}{\sqrt{\log\left(D\right)}}\right)$.

###### Proof.

We have from standard $\chi^{2}$ tail bounds (see, e.g., Lemma 1 of Laurent and Massart []) that:

$$ $\displaystyle\mathbb{P}\left[\chi^{2}-D\geq 2D^{\frac{3}{4}}t+2\sqrt{D}t^{2}\right]$ $\displaystyle\leq\exp\left(-\sqrt{D}t^{2}\right),$ (335) $\displaystyle\mathbb{P}\left[\chi^{2}-D\leq-2D^{\frac{3}{4}}t\right]$ $\displaystyle\leq\exp\left(-\sqrt{D}t^{2}\right).$ (336) $$

This implies that:

$$ $\mathbb{P}\left[\chi\geq\sqrt{D+2D^{\frac{3}{4}}t+2\sqrt{D}t^{2}}\right]\leq \exp\left(-\sqrt{D}t^{2}\right)$ (337) $$

and (for $t\leq\frac{1}{2}D^{\frac{1}{4}}$)

$$ $\mathbb{P}\left[\chi\leq\sqrt{D-2D^{\frac{3}{4}}t}\right]\leq\exp\left(-\sqrt{ D}t^{2}\right).$ (338) $$

Using the general inequality (for $x\geq-1$):

$$ $\sqrt{1+x}\leq 1+\frac{x}{2},$ (339) $$

we have that:

$$ $\sqrt{D+2D^{\frac{3}{4}}t+2\sqrt{D}t^{2}}-\sqrt{D}\leq D^{\frac{1}{4}}t+t^{2}$ (340) $$

such that

$$ $\mathbb{P}\left[\chi-\sqrt{D}\geq D^{\frac{1}{4}}t+t^{2}\right]\leq\exp\left(- \sqrt{D}t^{2}\right).$ (341) $$

Similarly, using the general inequality (for $0\leq x\leq\frac{3}{4}$):

$$ $1-\frac{2x}{3}\leq\sqrt{1-x},$ (342) $$

we have under the given assumptions that:

$$ $\mathbb{P}\left[\sqrt{D}-\chi\geq\frac{4}{3}D^{\frac{1}{4}}t\right]\leq\exp \left(-\sqrt{D}t^{2}\right).$ (343) $$

Noting that both $D^{\frac{1}{4}}t+t^{2}$ and $\frac{4}{3}D^{\frac{1}{4}}t$ are upper bounded by $\frac{4}{3}D^{\frac{1}{4}}t+t^{2}$ then implies the final result.
∎

We end with a tail bound for the off-diagonal entries of a $\beta$-Wishart matrix.

###### Lemma 43 (Off-diagonal $\beta$ -Wishart tail bounds) .

Let $\bm{W}$ be a $\left(p+1\right)\times\left(p+1\right)$ $\beta$-Wishart matrix with $r$ degrees of freedom and identity scale matrix. There exist universal constants $C,K>0$ that depends only on $\beta$ such that, for every $t\geq 0$,

$$ $\mathbb{P}\left[r^{-\frac{1}{2}}\max_{i\in\left[1,\ldots,p\right]}\left\lvert \bra{0}\bm{W}\ket{i}\right\rvert\geq 1+t\right]\leq Kp\exp\left(-C\min\left(t, t^{2}\right)\right).$ (344) $$

###### Proof.

Let $\bm{L}\bm{L}^{\dagger}$ be the Bartlett decomposition of $\bm{W}$. Let $\chi_{0}$ be the $\left(0,0\right)$ entry of $\bm{L}$ and $N_{0,i}$ the $\left(i,0\right)$ entry. The lemma statement is then equivalent to showing that

$$ $\mathbb{P}\left[\chi_{0}^{2}\max_{i\in\left[1,\ldots,p\right]}\left\lvert N_{0 ,i}\right\rvert^{2}\geq r\left(1+t\right)^{2}\right]\leq Kp\exp\left(-C\min \left(t,t^{2}\right)\right).$ (345) $$

Note that:

$$ $\displaystyle\mathbb{P}\left[\chi_{0}^{2}\max_{i\in\left[1,\ldots,p\right]} \left\lvert N_{0,i}\right\rvert^{2}\geq r\left(1+t\right)^{2}\right]$ $\displaystyle\leq\mathbb{P}\left[\max\left(\frac{\chi_{0}^{2}}{r},\max_{i\in \left[1,\ldots,p\right]}\left\lvert N_{0,i}\right\rvert^{2}\right)^{2}\geq \left(1+t\right)^{2}\right]$ (346) $\displaystyle=\mathbb{P}\left[\max\left(\frac{\chi_{0}^{2}}{r},\max_{i\in\left [1,\ldots,p\right]}\left\lvert N_{0,i}\right\rvert^{2}\right)\geq 1+t\right],$ $$

so by the union bound we need only check whether

$$ $\max\left(\mathbb{P}\left[\chi_{0}^{2}\geq r\left(1+t\right)\right],\mathbb{P} \left[\left\lvert N_{0,i}\right\rvert^{2}\geq 1+t\right]\right)\leq\frac{K}{2} \exp\left(-C\min\left(t,t^{2}\right)\right)$ (347) $$

to prove the final result. As $\beta\left\lvert N_{0,i}\right\rvert^{2}$ is $\chi^{2}$-distributed with $\beta$ degrees of freedom the final result immediately follows from standard $\chi^{2}$ tail bounds.
∎

###### Lemma 41 (Robustness of β 𝛽 \beta italic_β -Wishart Bartlett decompositions) .

###### Proof.

###### Lemma 42 (Tail bounds for χ 𝜒 \chi italic_χ -distributed random variables) .

###### Proof.

###### Lemma 43 (Off-diagonal β 𝛽 \beta italic_β -Wishart tail bounds) .

###### Proof.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 8
presentation: 3
questions: Can you give more explanation and background on N (equation 25) in the main text?
soundness: 4
strengths: - The article is very well structured and seems technically of excellent quality. (However, I did not check or verify the proofs in the Appendix in detail) 
- The overall question on the structure of quantum neural networks and their potential advantages is relevant and the new findings advance the field.
The new Wishart process framework introduced in this article seems useful, as demonstrated by the results derived mentioned in the summary.
summary: The article defines a quantum neural network (QNN) as a chain of unitary operations applied to an initial quantum state. The loss function is the expectation value of an operator the resulting state with respect to an operator. This definition is common and general. The authors then assume random initialization and show that the loss converges to what they term a Wishart process in the limit of infinitely wide networks. This is in contrast to classical neural nets, where the limit is a Gaussian process (GP).   Having established the Wishart process framework, they use it to formulate several results within it:

(1) While the possibility of QNN convergence to GPs had already been studied in prior cited work, the authors formulate conditions within this framework for the convergence.

(2) Furthermore, they extend and generalize two results on trainability of neural networks from cited articles: 
- Conditions for barren plateaus, i.e. vanishing gradients.
- The density of local minima is concentrated near the global minimum iff the network is over-parameterized.
weaknesses: - The results are not groundbreaking new but generalizations of previous results and formulated in a more general language and setting.
- The results are a bit underwhelming, essentially "it seems unlikely that there exists any computational quantum advantage during the training of QNNs", which already seemed unlikely before. But this is expected, and this evidence is also valuable, so this is not a strong weakness.
- The main text is hard to understand without the long Appendix. However, this seems inevitable given the page limit and the subject.

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: * Where does entanglement fall into these results? One criticism of quantum ML is that entanglement appears not to provide much a benefit in real-world problems. Is there an element of the theory presented here that can touch on this problem?

* What exactly should follow a Wishart distribution in the QNN framework? In a neural network, it is the weights, whereas the NTK will likely look Wishart and the Fisher matrix the same. Purely looking at the math, the NTK and Fisher matrix of a QNN should do the same.
soundness: 2
strengths: The paper is written very well and tackles a valid problem. The theoretical analysis appears to be well performed and the authors have done a lot to highlight the connection to prior results.
summary: The authors present a theory study of quantum neural networks showing, in contrast to classical neural network studies, that QNNs obey a so-called Wishart process. In doing so, they derive the existence of barren plateaus and show when a QNN behaves like a Gaussian process.
weaknesses: The main weakness I see is in the discussion around limits. It isn't always clear in what limit the authors are discussing the behaviour of the networks. It is also reasonable that a classical neural network's neural tangent kernel matrix has eigenvalues behaving a Wishart distribution and evolving during training. Further, a covariance matrix used to describe a GPR will likely also look very Wishart. While the math and results were presented in the paper, it was never completely clear to me how this all aligned and how I should think about the WIshart process.

---

confidence: 3
contribution: 4
initial_rating: 3
presentation: 1
questions: what is the potential of classical limits within this framework?
soundness: 2
strengths: 1) Several new ideas are introduced from theoretical physics / quantum mechanics and connected with neural networks

2) The claimed result is a major generalization of major theorems in "classical" neural networks such as the infinite-width limit of neural networks (Gaussian processes) or Neural Tangent Kernel theory. While several results are derived from this for quantum NNs, there is potential that these ideas can also lead to new results for classical NNs, e.g. via the correspondence principle.
summary: It is known that (classical) neural networks typically behave like Gaussian processes in the infinite-width limit under reasonable regularity assumptions. In this work, this result is extended to Quantum Neural Networks which behave like a Wishart process instead. This result is then used to study training, similar to Neural Tangent Kernels in the classical case, and the loss landscape thereof in particular with respect to barren plateus.

While the authors should be commended on their opus magnum and an exciting set of ideas / contributions from theoretical physics to machine learning, I share some concerns regarding technical soundness, presentation and topical fit to this publication venue.
weaknesses: 1) Technical: The paper lacks basic definitions and introduction of notation, or precision in doing so, that are crucial to follow the train of thought. This is detrimental as several parts of the notation are non-standard, or at least non-standard within the subsections and audiences of this publication venue. For example, in Theorem 1, a notation is introduced as "....denotes the projection into a (Jordan sub-algebra) A_α ....", first mention in the introduction around eq 2. Nowhere in the main body is it defined what this precisely means. QM people are left with a vague idea, but not more. Instead, the reader must dig deep into the appendix. It was until around eq 178 in the appendix until I got a better idea, by implication only. Further examples of lack of definition are around eq 5, 7 and 13, or mathematical objects \mathcal{G} and \mathcal{H} in Line 291. An example for unclear notation is also in eq 21 in Theorem 2. While non-standard notation is perfectly fine, it doesn't help much if things are kept too informal, even when things "are easy to see", and it should defined and introduced properly. This list is non-exhaustive.

2) Presentation: When agreeing to review, I was expecting a 10-page paper in line with the ICLR submission guidelines. While this work technically fulfills these requirements, one can learn and understand very little without working through the 45 page appendix. More honestly, this is a 59-page paper that has been last minute broken into a 10 page teaser with several big claims, many with unclear or vague rationale, followed by the actual paper.

Minor:
I know this publication venue typically expects numerical experiments, of which there is a lack of any in this paper. From a theoretical physics perspective, this is fine though.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Is there an algorithm capable of automatically determining the trainability of any given QNNs?

2. Could you explain how to calculate $O^\alpha$? Are there any efficient algorithms for evaluating these calculations? It would be better if you can provide an example calculation or pseudocode for computing $O^\alpha$.

3. Previous work in [Anschuetz & Kiani,2022] analyzes local minima phenomena in local and deep, brickwall circuits. Does the current study address these issues? Furthermore, is it applicable to shallow circuits as well?

4. Does this work cover all previous results concerning BP and local minima?  If possible, it would be better to provide a more comprehensive comparison table or discussion section that explicitly shows how their framework subsumes or extends specific prior results on barren plateaus and local minima.

5. Could you clarify the rationale behind using Jordan algebra in this study?
soundness: 4
strengths: - This paper establishes a unified framework for quantum nerual networks.

- It shows the necessary and sufficient conditions for QNNs to approach the Gaussian process limit, offering comprehensive insights into the trainability of these networks.

- While Jordan algebra is relatively unfamiliar in this domain, the authors have made efforts to clarify its application and relevance to the audience.

Overall, the results presented in this paper are impressive, and I believe they make a valuable contribution to the field of quantum machine learning.
summary: This work establishes a unified theoretical framework for characterizing the trainability behavior of quantum neural networks. It integrates previous findings on barren plateaus and local minima phenomena by introducing a novel class of random processes.
weaknesses: - The paper should clearly show its contributions by more explicitly comparing what it offers beyond previous works. For example, it would be beneficial to clarify whether the results offer improvements in handling shallow circuits compared to previous studies in local minima.

Meta-review: The current paper proves that the loss and its first two derivatives of quantum neural networks asymptotically form Wishart processes. The reviewers acknowledged that it is a solid contribution extending classical results with deep theoretical insights. The main criticisms arise from the readability of the main text, based on which the authors made a positive effort in the rebuttal phase. There is still notable place for improvements in the self-containedness after re-shuffling the contents and the overall clarity, e.g., on the concept of "ansatz" and definition 7 in the conclusion section.

In the final version, please consider addressing all reviewer's comments especially the criticisms raised by reviewer 3K5s, and make sure the main definitions and theorems are clearly presented and self-contained.

Decision: Accept (Poster)