SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\partial_ {\mu} \mathcal {L} = - \langle \phi | D _ {\mu} | \phi \rangle = - p ^ {2} \langle \psi^ {*} | D _ {\mu} | \psi^ {*} \rangle - q ^ {2} \langle \psi^ {\perp} | D _ {\mu} | \psi^ {\perp} \rangle - 2 p q \operatorname {R e} \left(\langle \psi^ {\perp} | D _ {\mu} | \psi^ {*} \rangle\right). \tag {S43}
$$
where  $q = \sqrt{1 - p^2}$  and  $D_{\mu} = \partial_{\mu}\varrho$  is a traceless Hermitian operator since  $\operatorname {tr}D_{\mu} = \partial_{\mu}(\operatorname {tr}\varrho) = 0$  At  $\pmb {\theta} = \pmb{\theta}^{*}$  , the operator  $D_{\mu}$  is reduced to  $D_{\mu}^{*}$  which satisfies several useful properties
$$
D _ {\mu} ^ {*} = \left[ \partial_ {\mu} \left(| \psi \rangle \langle \psi |\right) \right] ^ {*} = \left| \partial_ {\mu} \psi^ {*} \rangle \langle \psi^ {*} \right| + | \psi^ {*} \rangle \langle \partial_ {\mu} \psi^ {*} |,
$$
$$
\langle \psi^ {*} | D _ {\mu} ^ {*} | \psi^ {*} \rangle = \langle \psi^ {*} | \partial_ {\mu} \psi^ {*} \rangle + \langle \partial_ {\mu} \psi^ {*} | \psi^ {*} \rangle = \partial_ {\mu} (\langle \psi | \psi \rangle) ^ {*} = 0, \tag {S44}
$$
$$
\langle \psi^ {\perp} | D _ {\mu} ^ {*} | \psi^ {\perp} \rangle = \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle \langle \psi^ {*} | \psi^ {\perp} \rangle + \langle \psi^ {\perp} | \psi^ {*} \rangle \langle \partial_ {\mu} \psi^ {*} | \psi^ {\perp} \rangle = 0, \tag {S44}
$$
$$
\langle \psi^ {\perp} | D _ {\mu} ^ {*} | \psi^ {*} \rangle = \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle \langle \psi^ {*} | \psi^ {*} \rangle + \langle \psi^ {\perp} | \psi^ {*} \rangle \langle \partial_ {\mu} \psi^ {*} | \psi^ {*} \rangle = \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle ,
$$
where we have used the facts of  $\langle \psi |\psi \rangle = 1$  and  $\langle \psi^{*}|\psi^{\perp}\rangle = 0$ . Note that  $|\partial_{\mu}\psi^{*}\rangle$  actually refers to  $(\partial_{\mu}|\psi \rangle)^{*}$ . Thus the variance of the first order derivative at  $\theta = \theta^{*}$  becomes
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \mathcal {L} ^ {*} \right] = \mathbb {E} _ {\mathbb {T}} \left[ \left(\partial_ {\mu} \mathcal {L} ^ {*} - \mathbb {E} _ {\mathbb {T}} \left[ \partial_ {\mu} \mathcal {L} ^ {*} \right]\right) ^ {2} \right] = \mathbb {E} _ {\mathbb {T}} \left[ \left(\partial_ {\mu} \mathcal {L} ^ {*}\right) ^ {2} \right] \tag {S45} \\ = 4 p ^ {2} q ^ {2} \mathbb {E} _ {\mathbb {T}} \left[ \left(\operatorname {R e} \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle\right) ^ {2} \right]. \\ \end{array}
$$
According to Lemma S2 and Corollary S4, it holds that
$$
\begin{array}{l} \mathbb {E} _ {\mathbb {T}} \left[ \left(\operatorname {R e} \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle\right) ^ {2} \right] = \frac {1}{2} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | \partial_ {\mu} \psi^ {*} \rangle \langle \partial_ {\mu} \psi^ {*} | \psi^ {\perp} \rangle \right] \tag {S46} \\ = \frac {\left\langle \partial_ {\mu} \psi^ {*} \mid \partial_ {\mu} \psi^ {*} \right\rangle - \left\langle \psi^ {*} \mid \partial_ {\mu} \psi^ {*} \right\rangle \left\langle \partial_ {\mu} \psi^ {*} \mid \psi^ {*} \right\rangle}{2 (d - 1)} = \frac {\mathcal {F} _ {\mu \mu} ^ {*}}{4 (d - 1)}, \\ \end{array}
$$
where  $\mathcal{F}_{\mu \mu}$  is the QFI diagonal element. Using the generators in the PQC,  $\mathcal{F}_{\mu \mu}$  could be expressed as
$$
\mathcal {F} _ {\mu \mu} = 2 \left(\langle \psi | \tilde {\Omega} _ {\mu} ^ {2} | \psi \rangle - \langle \psi | \tilde {\Omega} _ {\mu} | \psi \rangle^ {2}\right), \tag {S47}
$$
where  $\tilde{\Omega}_{\mu} = V_{\mu \rightarrow M}\Omega_{\mu}V_{\mu \rightarrow M}^{\dagger}$ . Finally, the variance of  $\partial_{\mu}\mathcal{L}$  at  $\pmb {\theta} = \pmb{\theta}^{*}$  equals to
$$
\operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \mathcal {L} ^ {*} \right] = 4 p ^ {2} q ^ {2} \cdot \frac {\mathcal {F} _ {\mu \mu} ^ {*}}{4 (d - 1)} = \frac {p ^ {2} \left(1 - p ^ {2}\right)}{d - 1} \mathcal {F} _ {\mu \mu} ^ {*}. \tag {S48}
$$
The second order derivative can be expressed by
$$
\begin{array}{l} \left(H _ {\mathcal {L}}\right) _ {\mu \nu} = \frac {\partial^ {2} \mathcal {L}}{\partial \theta_ {\mu} \partial \theta_ {\nu}} = \partial_ {\mu} \partial_ {\nu} \mathcal {L} = - \langle \phi | D _ {\mu \nu} | \phi \rangle \tag {S49} \\ = - p ^ {2} \langle \psi^ {*} | D _ {\mu \nu} | \psi^ {*} \rangle - q ^ {2} \langle \psi^ {\perp} | D _ {\mu \nu} | \psi^ {\perp} \rangle - 2 p q \operatorname {R e} \left(\langle \psi^ {\perp} | D _ {\mu \nu} | \psi^ {*} \rangle\right), \\ \end{array}
$$
where  $D_{\mu \nu} = \partial_{\mu}\partial_{\nu}\varrho$  is a traceless Hermitian operator since  $\mathrm{tr}D_{\mu \nu} = \partial_{\mu}\partial_{\nu}(\mathrm{tr}\underline{\varrho}) = 0$ . Please do not confuse  $D_{\mu \nu}$  with  $D_{\mu}$  above. At  $\pmb {\theta} = \pmb{\theta}^{*}$ , the  $D_{\mu \nu}$  is reduced to  $D_{\mu \nu}^{*}$  which satisfies the following properties
$$
D _ {\mu \nu} ^ {*} = \partial_ {\mu} \partial_ {\nu} (| \psi \rangle \langle \psi |) ^ {*} = 2 \operatorname {R e} \left[ | \partial_ {\mu} \partial_ {\nu} \psi^ {*} \rangle \langle \psi^ {*} | + | \partial_ {\mu} \psi^ {*} \rangle \langle \partial_ {\nu} \psi^ {*} | \right], \tag {S50}
$$
$$
\langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle = 2 \operatorname {R e} \left[ \langle \psi^ {*} | \partial_ {\mu} \partial_ {\nu} \psi^ {*} \rangle + \langle \psi^ {*} | \partial_ {\mu} \psi^ {*} \rangle \langle \partial_ {\nu} \psi^ {*} | \psi^ {*} \rangle \right] = - \mathcal {F} _ {\mu \nu}, \tag {S51}
$$
$$
\langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle = 2 \operatorname {R e} \left[ \langle \psi^ {\perp} | \partial_ {\nu} \psi^ {*} \rangle \langle \partial_ {\mu} \psi^ {*} | \psi^ {\perp} \rangle \right]. \tag {S52}
$$
Here the notation  $2\operatorname{Re}[\cdot]$  of square matrix  $A$  actually means the sum of the matrix and its Hermitian conjugate, i.e.,  $2\operatorname{Re}[A] = A + A^{\dagger}$ . From Eq. (S50) we know that the rank of  $D_{\mu \nu}$  is at most 4. Substituting the expectation in Eq. (S42), the variance of the second order derivative at  $\theta = \theta^{*}$  becomes
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right] = \mathbb {E} _ {\mathbb {T}} \left[ \left(\partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} - \mathbb {E} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right]\right) ^ {2} \right] \\ = \left(\frac {q ^ {2}}{d - 1} \mathcal {F} _ {\mu \nu} ^ {*}\right) ^ {2} + q ^ {4} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle^ {2} \right] \tag {S53} \\ - \frac {2 q ^ {4}}{d - 1} \mathcal {F} _ {\mu \nu} ^ {*} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle \right] + 4 p ^ {2} q ^ {2} \mathbb {E} _ {\mathbb {T}} \left[ (\mathrm {R e} \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle) ^ {2} \right]. \\ \end{array}
$$
where the inhomogeneous cross terms vanish after taking the expectation according to Lemma S2 and have been omitted. Using Corollaries S4 and S6, the expectations in Eq. (S53) can be calculated as
$$
\mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle^ {2} \right] = \frac {\mathrm {t r} ((D _ {\mu \nu} ^ {*}) ^ {2}) - 2 \left(\langle \psi^ {*} | (D _ {\mu \nu} ^ {*}) ^ {2} | \psi^ {*} \rangle - \langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle^ {2}\right)}{d (d - 1)},
$$
$$
\mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle \right] = - \frac {\langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle}{d - 1} = \frac {\mathcal {F} _ {\mu \nu} ^ {*}}{d - 1}, \tag {S54}
$$
$$
\begin{array}{l} \mathbb {E} _ {\mathbb {T}} \left[ (\mathrm {R e} \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle) ^ {2} \right] = \frac {1}{2} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle \langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {\perp} \rangle \right] \\ = \frac {\langle \psi^ {*} | (D _ {\mu \nu} ^ {*}) ^ {2} | \psi^ {*} \rangle - \langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle^ {2}}{2 (d - 1)}. \\ \end{array}
$$
Thus the variance of the second order derivative at  $\theta = \theta^{*}$  can be written as
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right] = q ^ {4} \frac {\| D _ {\mu \nu} ^ {*} \| _ {2} ^ {2} - 2 \left(\langle \psi^ {*} | (D _ {\mu \nu} ^ {*}) ^ {2} | \psi^ {*} \rangle - \langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle^ {2}\right)}{d (d - 1)} \tag {S55} \\ + 2 p ^ {2} q ^ {2} \frac {\langle \psi^ {*} | (D _ {\mu \nu} ^ {*}) ^ {2} | \psi^ {*} \rangle - \langle \psi^ {*} | D _ {\mu \nu} ^ {*} | \psi^ {*} \rangle^ {2}}{d - 1} - \left(\frac {q ^ {2}}{d - 1} \mathcal {F} _ {\mu \nu} ^ {*}\right) ^ {2}. \\ \end{array}
$$
Note that the factor
$$
\left\langle \psi^ {*} \right| \left(D _ {\mu \nu} ^ {*}\right) ^ {2} \left| \psi^ {*} \right\rangle - \left\langle \psi^ {*} \right| D _ {\mu \nu} ^ {*} \left| \psi^ {*} \right\rangle^ {2} = \left\langle \psi^ {*} \right| D _ {\mu \nu} ^ {*} (I - \left| \psi^ {*} \right\rangle \left\langle \psi^ {*} \right|) D _ {\mu \nu} ^ {*} \left| \psi^ {*} \right\rangle , \tag {S56}
$$
is non-negative because the operator  $(I - |\psi^{*}\rangle \langle \psi^{*}|)$  is positive semidefinite. Hence the variance can be upper bounded by
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right] \leq \frac {q ^ {4}}{d (d - 1)} \| D _ {\mu \nu} ^ {*} \| _ {2} ^ {2} + \frac {2 p ^ {2} q ^ {2}}{d - 1} \langle \psi^ {*} | (D _ {\mu \nu} ^ {*}) ^ {2} | \psi^ {*} \rangle \\ \leq \frac {q ^ {4}}{d (d - 1)} \| D _ {\mu \nu} ^ {*} \| _ {2} ^ {2} + \frac {2 p ^ {2} q ^ {2}}{d - 1} \| \left(D _ {\mu \nu} ^ {*}\right) ^ {2} \| _ {\infty} - \left(\frac {q ^ {2}}{d - 1} \mathcal {F} _ {\mu \nu} ^ {*}\right) ^ {2} \tag {S57} \\ \leq \frac {2 q ^ {2}}{d - 1} \left(p ^ {2} + \frac {2 q ^ {2}}{d}\right) \| D _ {\mu \nu} ^ {*} \| _ {\infty} ^ {2}, \\ \end{array}
$$
where we have used the properties
$$
\left\| D _ {\mu \nu} ^ {*} \right\| _ {2} \leq \sqrt {\operatorname {r a n k} \left(D _ {\mu \nu} ^ {*}\right)} \left\| D _ {\mu \nu} ^ {*} \right\| _ {\infty} \leq 2 \left\| D _ {\mu \nu} ^ {*} \right\| _ {\infty}, \quad \left\| \left(D _ {\mu \nu} ^ {*}\right) ^ {2} \right\| _ {\infty} = \left\| D _ {\mu \nu} ^ {*} \right\| _ {\infty} ^ {2}. \tag {S58}
$$
Utilizing the quantum gates in the QNN, the operator  $D_{\mu \nu}$  can be written as
$$
D _ {\mu \nu} = V _ {\nu + 1 \rightarrow M} \left[ V _ {\mu + 1 \rightarrow \nu} \left[ V _ {1 \rightarrow \mu} | 0 \rangle \langle 0 | V _ {1 \rightarrow \mu} ^ {\dagger}, i \Omega_ {\mu} \right] V _ {\mu + 1 \rightarrow \nu} ^ {\dagger}, i \Omega_ {\nu} \right] V _ {\nu + 1 \rightarrow M} ^ {\dagger}, \tag {S59}
$$
where we assume  $\mu \leq \nu$  without loss of generality. Thus  $\| D_{\mu \nu}\|_{\infty}$  can be upper bounded by
$$
\left\| D _ {\mu \nu} \right\| _ {\infty} \leq 4 \left\| \Omega_ {\mu} \Omega_ {\nu} \right\| _ {\infty} \leq 4 \left\| \Omega_ {\mu} \right\| _ {\infty} \left\| \Omega_ {\nu} \right\| _ {\infty}. \tag {S60}
$$
Finally, the variance of the second order derivative at  $\theta = \theta^{*}$  can be bounded as
$$
\operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right] \leq f _ {2} (p, d) \| \Omega_ {\mu} \| _ {\infty} ^ {2} \| \Omega_ {\nu} \| _ {\infty} ^ {2}. \tag {S61}
$$
The factor  $f_{2}(p,d)$  reads
$$
f _ {2} (p, d) = \frac {3 2 \left(1 - p ^ {2}\right)}{d - 1} \left[ p ^ {2} + \frac {2 \left(1 - p ^ {2}\right)}{d} \right], \tag {S62}
$$
which vanishes at least of order  $1 / d$ .
Note that when  $p \in \{0,1\}$ ,  $f_{1}, f_{2}$  and hence the variances of the first and second order derivatives become exactly zero, indicating  $\mathcal{L}^*$  takes the optimum in all cases. This is nothing but the fact that the range of the loss function is [0, 1], which reflects that the bound of  $\mathrm{Var}_{\mathbb{T}}[\partial_{\mu}\partial_{\nu}\mathcal{L}^{*}]$  is tight in  $p$ .
We remark that the vanishing gradient here is both conceptually and technically distinct from barren plateaus [35]. Firstly, here we focus on a fixed parameter point  $\theta^{*}$  instead of a randomly chosen point on the training landscape. Other points apart from  $\theta^{*}$  is allowed to have a non-vanishing gradient expectation, which leads to prominent local minima instead of plateaus. Moreover, the ensemble  $\mathbb{T}$  used here originates from the unknown target state instead of the random initialization. The latter typically demands a polynomially deep circuit to form a 2-design. Technically, a constant overlap  $p$  is assumed to construct the ensemble  $\mathbb{T}$  instead of completely random over the entire Hilbert space. Thus our results apply to adaptive methods, while barren plateaus from the random initialization are not.
$$
\Pr_ {\mathbb {T}} [ \neg \operatorname {L o c a l M i n} (\boldsymbol {\theta} ^ {*}, \epsilon) ] \leq \frac {2 f _ {1} (p , d) \| \boldsymbol {\omega} \| _ {2} ^ {2}}{\epsilon_ {1} ^ {2}} + \frac {f _ {2} (p , d) \| \boldsymbol {\omega} \| _ {2} ^ {4}}{\left(\frac {d p ^ {2} - 1}{d - 1} e ^ {*} + \epsilon_ {2}\right) ^ {2}}, \tag {S63}
$$
where  $e^{*}$  denotes the minimal eigenvalue of the QFI matrix at  $\pmb{\theta} = \pmb{\theta}^{*}$ .  $f_{1}$  and  $f_{2}$  are defined in Lemma 1 which vanish at least of order  $1 / d$ .
Proof By definition in Eq. (5) in the main text, the probability  $\mathrm{Pr}_{\mathbb{T}}[\neg \mathrm{LocalMin}(\pmb{\theta}^*, \epsilon)]$  can be upper bounded by the sum of two terms: the probability that one of the gradient component is larger than
$\epsilon_{1}$ , and the probability that the Hessian matrix is not positive definite up to the error  $\epsilon_{2}$ , i.e.,
$$
\begin{array}{l} \Pr_ {\mathbb {T}} [ \neg \operatorname {L o c a l M i n} (\boldsymbol {\theta} ^ {*}, \epsilon) ] = \Pr_ {\mathbb {T}} \left[ \bigcup_ {\mu = 1} ^ {M} \left\{\left| \partial_ {\mu} \mathcal {L} ^ {*} \right| > \epsilon_ {1} \right\} \cup \left\{H _ {\mathcal {L}} ^ {*} \neq - \epsilon_ {2} I \right\} \right] \tag {S64} \\ \leq \Pr_ {\mathbb {T}} \left[ \bigcup_ {\mu = 1} ^ {M} \left\{\left| \partial_ {\mu} \mathcal {L} ^ {*} \right| > \epsilon_ {1} \right\} \right] + \Pr_ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \not \succ - \epsilon_ {2} I \right]. \\ \end{array}
$$
The first term can be easily upper bounded by combining Lemma 1 and Chebyshev's inequality, i.e.,
$$
\Pr_ {\mathbb {T}} \left[ \bigcup_ {\mu = 1} ^ {M} \left\{\left| \partial_ {\mu} \mathcal {L} ^ {*} \right| > \epsilon_ {1} \right\} \right] \leq \sum_ {\mu = 1} ^ {M} \Pr_ {\mathbb {T}} \left[ \left| \partial_ {\mu} \mathcal {L} ^ {*} \right| > \epsilon_ {1} \right] \leq \sum_ {\mu = 1} ^ {M} \frac {\operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \mathcal {L} ^ {*} \right]}{\epsilon_ {1} ^ {2}} = \frac {f _ {1} (p , d)}{\epsilon_ {1} ^ {2}} \operatorname {t r} \mathcal {F} ^ {*}, \tag {S65}
$$
where the diagonal element of the QFI matrix is upper bounded as  $\mathcal{F}_{\mu \mu} \leq 2\|\Omega_{\mu}\|_{\infty}^{2}$  by definition and thus  $\operatorname{tr} \mathcal{F}^{*} \leq 2\|\omega\|_{2}^{2}$ . Here the generator norm vector  $\omega$  is defined as
$$
\boldsymbol {\omega} = \left(\| \Omega_ {1} \| _ {\infty}, \| \Omega_ {2} \| _ {\infty}, \dots , \| \Omega_ {M} \| _ {\infty}\right), \tag {S66}
$$
so that the squared vector 2-norm of  $\omega$  equals to  $\| \omega \| _2^2 = \sum_{\mu = 1}^{M}\| \Omega_\mu \|_\infty^2$ . Thus we obtain the upper bound of the first term, i.e.,
$$
\Pr_ {\mathbb {T}} \left[ \bigcup_ {\mu = 1} ^ {M} \left\{\left| \partial_ {\mu} \mathcal {L} ^ {*} \right| > \epsilon_ {1} \right\} \right] \leq \frac {2 f _ {1} (p , d) \| \boldsymbol {\omega} \| _ {2} ^ {2}}{\epsilon_ {1} ^ {2}}, \tag {S67}
$$
It takes extra efforts to bound the second term. After assuming  $p^2 > 1 / d$  to ensure that  $\mathbb{E}_{\mathbb{T}}[H_{\mathcal{L}}^{*}]$  is positive semidefinite, a sufficient condition of the positive definiteness can be obtained by perturbing  $\mathbb{E}_{\mathbb{T}}[H_{\mathcal{L}}^{*}]$  using Lemma S8, i.e.,
$$
\left\| H _ {\mathcal {L}} ^ {*} - \mathbb {E} _ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \right] \right\| _ {\infty} <   \left\| \mathbb {E} _ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} + \epsilon_ {2} I \right] ^ {- 1} \right\| _ {\infty} ^ {- 1} \quad \Rightarrow \quad H _ {\mathcal {L}} ^ {*} + \epsilon_ {2} I \succ 0, \tag {S68}
$$
Note that  $\| \mathbb{E}_{\mathbb{T}}[H_{\mathcal{L}}^{*} + \epsilon_{2}I]^{-1}\|_{\infty}^{-1} = \frac{dp^{2} - 1}{d - 1} e^{*} + \epsilon_{2}$ , where  $e^{*}$  denotes the minimal eigenvalue of the QFI  $\mathcal{F}^*$ . A necessary condition for  $H_{\mathcal{L}}^{*} + \epsilon_{2}I \neq 0$  is hence obtained by the contrapositive, i.e.,
$$
H _ {\mathcal {L}} ^ {*} \nrightarrow - \epsilon_ {2} I \quad \Rightarrow \quad \| H _ {\mathcal {L}} ^ {*} - \mathbb {E} _ {\mathbb {T}} [ H _ {\mathcal {L}} ^ {*} ] \| _ {\infty} \geq \frac {d p ^ {2} - 1}{d - 1} e ^ {*} + \epsilon_ {2}. \tag {S69}
$$
Thus the probability that  $H_{\mathcal{L}}^{*}$  is not positive definite can be upper bounded by
$$
\Pr_ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \succ - \epsilon_ {2} I \right] \leq \Pr_ {\mathbb {T}} \left[ \| H _ {\mathcal {L}} ^ {*} - \mathbb {E} _ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \right] \| _ {\infty} \geq \frac {d p ^ {2} - 1}{d - 1} e ^ {*} + \epsilon_ {2} \right]. \tag {S70}
$$
The generalized Chebyshev's inequality in Lemma S11 regarding  $H_{\mathcal{L}}^{*}$  and the Schatten-  $\infty$  norm gives
$$
\Pr_ {\mathbb {T}} \left[ \| H _ {\mathcal {L}} ^ {*} - \mathbb {E} _ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \right] \| _ {\infty} \geq \varepsilon \right] \leq \frac {\sigma_ {\infty} ^ {2}}{\varepsilon^ {2}}, \tag {S71}
$$
where the "norm variance" is defined as  $\sigma_{\infty}^{2} = \mathbb{E}_{\mathbb{T}}[||H_{\mathcal{L}}^{*} - \mathbb{E}_{\mathbb{T}}[H_{\mathcal{L}}^{*}]||_{\infty}^{2}]$ . By taking  $\varepsilon = \frac{dp^2 - 1}{d - 1} e^* + \epsilon_2$  we obtain
$$
\Pr_ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \ngeq - \epsilon_ {2} I \right] \leq \frac {\sigma_ {\infty} ^ {2}}{\left(\frac {d p ^ {2} - 1}{d - 1} e ^ {*} + \epsilon_ {2}\right) ^ {2}}. \tag {S72}
$$
Utilizing Lemma 1,  $\sigma_{\infty}^{2}$  can be further bounded by
$$
\begin{array}{l} \sigma_ {\infty} ^ {2} \leq \sigma_ {2} ^ {2} = \mathbb {E} _ {\mathbb {T}} [ \| H _ {\mathcal {L}} ^ {*} - \mathbb {E} _ {\mathbb {T}} [ H _ {\mathcal {L}} ^ {*} ] \| _ {2} ^ {2} ] = \sum_ {\mu \nu} \mathbb {E} _ {\mathbb {T}} \left[ \left(\left(H _ {\mathcal {L}} ^ {*}\right) _ {\mu \nu} - \left(\mathbb {E} _ {\mathbb {T}} [ H _ {\mathcal {L}} ^ {*} ]\right) _ {\mu \nu}\right) ^ {2} \right] \\ = \sum_ {\mu , \nu = 1} ^ {M} \operatorname {V a r} _ {\mathbb {T}} \left[ \partial_ {\mu} \partial_ {\nu} \mathcal {L} ^ {*} \right] \leq f _ {2} (p, d) \sum_ {\mu , \nu = 1} ^ {M} \| \Omega_ {\mu} \| _ {\infty} ^ {2} \| \Omega_ {\nu} \| _ {\infty} ^ {2} \tag {S73} \\ = f _ {2} (p, d) \left(\sum_ {\mu = 1} ^ {M} \| \Omega_ {\mu} \| _ {\infty} ^ {2}\right) ^ {2} = f _ {2} (p, d) \| \boldsymbol {\omega} \| _ {2} ^ {4}. \\ \end{array}
$$
Combining Eqs. (S72) and (S73), we obtain the upper bound of the second term, i.e.,
$$
\Pr_ {\mathbb {T}} \left[ H _ {\mathcal {L}} ^ {*} \neq - \epsilon_ {2} I \right] \leq \frac {f _ {2} (p , d) \| \boldsymbol {\omega} \| _ {2} ^ {4}}{\left(\frac {d p ^ {2} - 1}{d - 1} e ^ {*} + \epsilon_ {2}\right) ^ {2}}. \tag {S74}
$$
Substituting the bounds for the first and second terms into Eq. (S64), one finally arrives at the desired upper bound for the probability that  $\theta^{*}$  is not a local minimum up to a fixed precision  $\epsilon = (\epsilon_{1},\epsilon_{2})$
Note that the conclusion can be generalized to the scenario of mixed states or noisy states easily using the mathematical tools we developed in Appendix A.1. For example, suppose that the output state of the QNN is  $\rho(\pmb{\theta})$  and the target state is  $|\phi\rangle$ . The loss function can be defined as the fidelity distance  $\mathcal{L}(\pmb{\theta}) = 1 - \langle \phi | \rho(\pmb{\theta}) | \phi \rangle$ . Utilizing Lemmas S3 and S7, similar results can be carried out by calculating the subspace Haar integration.
$$
\mathbb {E} _ {\mathbb {T}} [ \mathcal {L} (\boldsymbol {\theta}) ] = 1 - p ^ {2} + \frac {d p ^ {2} - 1}{d - 1} g (\boldsymbol {\theta}), \tag {S75}
$$
Proof The expression of the expectation  $\mathbb{E}_{\mathbb{T}}[\mathcal{L}]$  has already been calculated in Eq. (S41). Considering Lemma S2, the variance of the loss function is
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} [ \mathcal {L} ] = \mathbb {E} _ {\mathbb {T}} \left[ (\mathcal {L} - \mathbb {E} _ {\mathbb {T}} [ \mathcal {L} ]) ^ {2} \right] \\ = \mathbb {E} _ {\mathbb {T}} \left[ \left(\frac {1 - p ^ {2}}{d - 1} \left(\langle \psi^ {*} | \varrho | \psi^ {*} \rangle - 1\right) + q ^ {2} \langle \psi^ {\perp} | \varrho | \psi^ {\perp} \rangle + 2 p q \operatorname {R e} \left(\langle \psi^ {\perp} | \varrho | \psi^ {*} \rangle\right)\right) ^ {2} \right] \tag {S76} \\ = \frac {q ^ {4}}{(d - 1) ^ {2}} (\langle \psi^ {*} | \varrho | \psi^ {*} \rangle - 1) ^ {2} + \frac {2 q ^ {4}}{d - 1} (\langle \psi^ {*} | \varrho | \psi^ {*} \rangle - 1) \mathbb {E} _ {\mathbb {T}} [ \langle \psi^ {\perp} | \varrho | \psi^ {\perp} \rangle ] \\ + q ^ {4} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | \varrho | \psi^ {\perp} \rangle^ {2} \right] + 4 p ^ {2} q ^ {2} \mathbb {E} _ {\mathbb {T}} \left[ \operatorname {R e} \left(\langle \psi^ {\perp} | \varrho | \psi^ {*} \rangle\right) ^ {2} \right], \\ \end{array}
$$
where  $q = \sqrt{1 - p^2}$  and  $\varrho (\pmb {\theta}) = |\psi (\pmb {\theta})\rangle \langle \psi (\pmb {\theta})|$ . According to Corollaries S4 and S6, the terms above can be calculated as
$$
\mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | \varrho | \psi^ {\perp} \rangle \right] = \frac {1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle}{d - 1},
$$
$$
\begin{array}{l} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | \varrho | \psi^ {\perp} \rangle^ {2} \right] = \frac {\left(\operatorname {t r} (\varrho^ {2}) - 2 \langle \psi^ {*} | \varrho^ {2} | \psi^ {*} \rangle + \langle \psi^ {*} | \varrho | \psi^ {*} \rangle^ {2}\right) + (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle) ^ {2}}{d (d - 1)} \tag {S77} \\ = \frac {2 (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle) ^ {2}}{d (d - 1)}, \\ \end{array}
$$
$$
\mathbb {E} _ {\mathbb {T}} \left[ \mathrm {R e} \left(\langle \psi^ {\perp} | \varrho | \psi^ {*} \rangle\right) ^ {2} \right] = \frac {1}{2} \mathbb {E} _ {\mathbb {T}} \left[ \langle \psi^ {\perp} | \varrho | \psi^ {*} \rangle \langle \psi^ {*} | \varrho | \psi^ {\perp} \rangle \right] = \frac {1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle^ {2}}{2 (d - 1)}.
$$
Thus the variance of the loss function becomes
$$
\begin{array}{l} \operatorname {V a r} _ {\mathbb {T}} [ \mathcal {L} ] = - \frac {q ^ {4} (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle) ^ {2}}{(d - 1) ^ {2}} + \frac {2 q ^ {4} (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle) ^ {2}}{d (d - 1)} + \frac {2 p ^ {2} q ^ {2} (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle^ {2})}{d - 1} \tag {S78} \\ = \frac {q ^ {2} (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle)}{d - 1} \left[ \frac {q ^ {2} (d - 2) (1 - \langle \psi^ {*} | \varrho | \psi^ {*} \rangle)}{d (d - 1)} + 2 p ^ {2} (1 + \langle \psi^ {*} | \varrho | \psi^ {*} \rangle) \right]. \\ \end{array}
$$
Substituting the relation  $\langle \psi^{*}|\varrho |\psi^{*}\rangle = 1 - g(\pmb {\theta})$ , the desired expression is obtained.