SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
D _ {\mathrm {K L}} \left(p _ {0} \| p _ {0, \hat {\boldsymbol {\theta}} _ {n} (\tau)}\right) \lesssim \left[ \frac {\tau^ {4}}{m n} + \frac {\tau^ {3}}{m ^ {2}} + \frac {1}{\tau} \right] + \left[ \frac {1}{m} + \bar {\tilde {\mathcal {L}}} (\bar {\boldsymbol {\theta}} ^ {*}) + \tilde {\mathcal {L}} (\boldsymbol {\theta} ^ {*}) \right] + D _ {\mathrm {K L}} \left(p _ {T} \| \pi\right), \quad \tau \geq 1,
$$
where  $\lesssim$  hides the term  $d\log (d + 1)$ , the polynomials of  $\log (1 / \delta^2)$ , finite RKHS norms and universal positive constants only depending on  $T$ .
Remark 2. Since  $p_0$  is compactly supported, the target score function  $s_0(\pmb{x}) = \nabla_{\pmb{x}} \log p_0(\pmb{x})$  is also defined on a compact domain. According to [12, 13],  $s_0$  is contained in the Barron function space with a finite Barron norm, and hence in a certain RKHS with a finite RKHS norm. Therefore, it is reasonable to require that the global minimizer  $s_{0,\theta^*}$  or  $\bar{s}_{0,\bar{\theta}^*}$  is also contained in some RKHS.
Proof sketch. Theorem 1 is proved via the following procedure.
1. According to Theorem 1 in [49], the KL divergence on the left-hand side can be upper bounded by the population loss of the trained model up to a small error. That is,
$$
D _ {\mathrm {K L}} \left(p _ {0} \| p _ {0, \hat {\boldsymbol {\theta}} _ {n} (\tau)}\right) \leq \tilde {\mathcal {L}} \left(\hat {\boldsymbol {\theta}} _ {n} (\tau); g ^ {2} (\cdot)\right) + D _ {\mathrm {K L}} \left(p _ {T} \| \pi\right). \tag {14}
$$
2. We use the model trained with respect to the population loss (10) to perform the decomposition:
$$
\begin{array}{l} \tilde {\mathcal {L}} (\hat {\boldsymbol {\theta}} _ {n} (\tau)) = \left[ \tilde {\mathcal {L}} (\hat {\boldsymbol {\theta}} _ {n} (\tau)) - \tilde {\mathcal {L}} (\boldsymbol {\theta} (\tau)) \right] + \tilde {\mathcal {L}} (\boldsymbol {\theta} (\tau)) \\ \lesssim \left[ \tilde {\mathcal {L}} \left(\hat {\boldsymbol {\theta}} _ {n} (\tau)\right) - \tilde {\mathcal {L}} (\boldsymbol {\theta} (\tau)) \right] + \tilde {\bar {\mathcal {L}}} (\bar {\boldsymbol {\theta}} (\tau)) + \text {M o n t e C a r l o} \triangleq I _ {1} + I _ {2} + I _ {3}, \tag {15} \\ \end{array}
$$
where we omit the weighting function  $g^{2}(\cdot)$  for simplicity. Here,  $\tilde{\mathcal{L}}$  is the loss objective obtained by replacing  $s_{t,\theta}$  in  $\tilde{\mathcal{L}}$  (defined in (7)) by  $\bar{s}_{t,\bar{\theta}}$ , and  $I_{3}$  summarizes the resulting Monte Carlo error.
3.  $I_{3}$  can be estimated via a similar argument as in [24] (Lemma 48).
4.  $I_{2}$  can be upper bounded via a standard analysis on the gradient flow dynamics over convex objectives.
5.  $I_{1}$  can be reduced as the norm product of  $s_{0,\hat{\pmb{\theta}}_n(\tau)}$ ,  $s_{0,\pmb {\theta}(\tau)}$  and their gap, then
(a) the former can be bounded with a square-root rate growth via a general norm estimate of parameters trained under the gradient flow dynamics;
(b) the latter can be estimated by the Rademacher complexity (see e.g., Chapter 26 in [43]).
Combining all above gives the desired result. The detailed proof is found in Appendix A.1.
Discussion on error bounds. The three error terms are further analyzed as follows.
- The first term is the main error, which implies an early-stopping generalization gap. In fact, if one selects an early-stopping time  $\tau_{\mathrm{es}}$  as  $\tau_{\mathrm{es}} = \Theta \left(n^{\frac{2}{5}}\right)$ , and let  $m \sim n$ , we have
$$
D _ {\mathrm {K L}} \left(p _ {0} \| p _ {0, \hat {\boldsymbol {\theta}} _ {n} (\tau_ {\mathrm {e s}})}\right) \lesssim (1 / n) ^ {\frac {2}{5}} + (1 / m) ^ {\frac {4}{5}}. \tag {16}
$$
- The second term is  $m$ -dependent and corresponds to the approximation error, which is  $o(1)$  when  $m \gg 1$ . In fact, the random feature model is a universal approximator to Lipschitz continuous functions on a compact domain (Theorem 6 in [17]). See more details in Appendix A.1 (the last paragraph).
- The third term is exponentially small in  $T$  since  $\pi$  (e.g. the Gaussian density) is log-Sobolev, according to a classical result in e.g. [55] (Theorem 3.20, Theorem 3.24 and Remark 3.26).
Remark 3. In practice, it is common to use the test error to evaluate the generalization performance. For diffusion models, a straightforward approach is to compute the negative log-likelihood (averaged in bits/dim; equivalent to the KL divergence) on the test dataset during training with the instantaneous change-of-variable formula ([8]) and probability flow ODE (defined in (4)), where the true score function  $\nabla_{\pmb{x}}\log p_t(\pmb{x})$  is replaced by  $s_{t,\theta (\tau)}(\pmb {x})$
Remark 4. Previous literature has established similar bounds for bias potential models ([66]) and GANs ([67]). Theorem 1 extends the corresponding results to the setting of diffusion models. Furthermore, this upper bound is finer in the sense that it incorporates the information regarding the model capacity (the hidden dimension  $m$  in this case), which shows that more parameters benefit the learning and generalization, as expected.
3.2.2 Data-Dependent Generalization Gap
In Section 3.2.1, we derive estimates on the generalization error for diffusion models along the training dynamics, where the target distribution is assumed to be finitely supported. In reality, this is often not the case, where target distributions usually possess distant multi-modes, from simple Gaussian mixtures to complicated Boltzmann distributions of physical systems ([29, 30]). Under these settings, the above analysis in Section 3.2.1 can not directly apply since the data domain is unbounded. It remains a problem to quantitatively characterize the generalization behavior of diffusion models given these target distributions with distant multi-modes or modes shift.
To provide a fine-grained demonstration of the generalization capability of diffusion models when applied to learn distributions with distant multi-modes, as an illustrating example, the Gaussian mixture with two modes is selected as the target distribution.
Theorem 2. Suppose the target distribution  $p_0$  is a one-dimensional 2-mode Gaussian mixture:  $p_0(x) = q_1\mathcal{N}(x; -\mu, 1) + q_2\mathcal{N}(x; \mu, 1)$ , where  $\mu > \sqrt{\log(1 / \delta^2)}$ ,  $q_1, q_2 > 0$  with  $q_1 + q_2 = 1$  are all constants. Under the conditions of Theorem 1 (except the uniform boundness of inputs), we have
$$
D _ {\mathrm {K L}} \left(p _ {0} \| p _ {0, \hat {\pmb {\theta}} _ {n} (\tau)}\right) \lesssim \mathrm {P o l y} (\mu) \left[ \frac {\tau^ {4}}{m n} + \frac {\tau^ {3}}{m ^ {2}} \right] + \frac {1}{\tau} + \left[ \frac {\mu^ {2}}{m} + \tilde {\bar {\mathcal {L}}} (\bar {\pmb {\theta}} ^ {*}) + \tilde {\mathcal {L}} (\pmb {\theta} ^ {*}) \right] + D _ {\mathrm {K L}} \left(p _ {T} \| \pi\right),
$$
where  $\tau \geq 1$ ,  $\lesssim$  hides the polynomials of  $\log(1/\delta^2)$ , finite RKHS norms and universal positive constants only depending on  $T$ .
Proof sketch. Theorem 2 is proved following a similar procedure with Theorem 1, except the input data  $x$  does not have a uniform bound here. This problem mainly affects the last step (5 (b)) in the proof sketch of Theorem 1, and can be handled by using the fact
$$
| x | \in [ \mu - \sqrt {\log (1 / \delta^ {2})}, \mu + \sqrt {\log (1 / \delta^ {2})} ] = \Theta (\mu) \tag {17}
$$
given the target Gaussian mixture distribution. The detailed proof is found in Appendix A.2.
Figure 2: An illustration of modes shift.
Remark 5. Theorem 2 indicates that, even for a simple target distribution (e.g. a one-dimensional 2-mode Gaussian mixture), the generalization error of diffusion models can be polynomially large regarding the modes' distance. Although Theorem 2 provides only an upper bound, the modes shift effect holds due to the model-target inconsistency (the last paragraph in Appendix A.2) and the following consistent experiments (Section 4.1.2).
4 Numerical Verifications
In this section, we numerically verify the previous theoretical results and insights (early-stopping generalization and modes shift effect) on both synthetic datasets and real-world datasets.
4.1 Simulations on Synthetic Datasets
4.1.1 Early-Stopping Generalization
First, we illustrate the early-stopping generalization gap established in Theorem 1. We select the one-hidden-layer neural network with Swish activations as the score network, which is trained using the SGD optimizer with a fixed learning rate 0.5. The target distribution is set to be a one-dimensional 2-mode Gaussian mixture with the modes' distance equaling 6, and the number of data samples is 1000. We measure the KL divergence from the trained model to the target distribution  $D_{\mathrm{KL}}\left(p_0\| p_{0,\hat{\theta}_n(\tau)}\right)$  along with the training epochs.
From Figure 3, one can observe that the KL divergence achieves its minimum at approximately the 800-th training epoch, and it starts to increase after this turning point. The experimental results are consistent with Theorem 1 (over multiple runs), which states that there exist early-stopping times when diffusion models can generalize well, indicating the effectiveness of our upper bound. Further, the KL divergence begins to oscillate after the minimum point, which may suggest a phase transition in the training dynamics, and the transition point is around the (optimal) early-stopping time.
Figure 3: The KL divergence dynamics.
4.1.2 Modes Shift Effect
Next, we numerically test the relationship between the modes' distance and generalization (density estimation) performance. All the configurations remain the same as Section 4.1.1, except that the target Gaussian mixtures have different modes' distances.
In Figure 4, the modeled distributions exhibit the following two-stage dynamics: (i) first gradually fitting the two modes (epoch  $= 100 \rightarrow 1000$ ); (ii) then diverging (epoch  $= 1000 \rightarrow 1900$ ). This aligns with the KL divergence dynamics (Figure 3) and again verifies the corresponding theoretical results (Theorem 1). However, Figure 5 shows that when the modes are distant from each other, there is difficulty in the learning process. In Figure 5, the optimal generalization is achieved at epoch  $= 100$ , but is still far from well generalizing. As the training proceeds, the learned model is almost always a single-mode distribution (epoch  $= 1000$ , 1900). This phenomenon is aligned with the results established in Theorem 2, which states that when there are distant modes in the target distribution, the generalization performance is relatively poor.
Figure 4: The training dynamics when the distance between two modes is  $6(\mu = 3)$
Figure 5: The training dynamics when the distance between two modes is  $30(\mu = 15)$ .
4.2 Simulations on Real-World Datasets
In this subsection, we verify our results on the MNIST dataset using the standard U-net architecture as the score network, which suggests that the adverse effect of modes shift on the generalization performance of diffusion models also appears in general.
The setup is as follows. First, we perform a  $K$ -means clustering on  $\mathcal{D}$  ( $\mathcal{D}$  denote the MNIST dataset) to get  $\mathcal{D} = \bigcup_{k=1}^{K} \mathcal{D}_k$ , and  $\bar{\pmb{x}}_k$  as the center of  $\mathcal{D}_k$ ,  $k = 1,2,\dots,K$ . Let  $(i^*,j^*) := \arg \max_{i \neq j} \| \bar{\pmb{x}}_i - \bar{\pmb{x}}_j \|_2$ , and  $\mathcal{D}_{\text{farthest}} := \mathcal{D}_{i^*} \bigcup \mathcal{D}_{j^*}$ .  $\mathcal{D}_{\text{nearest}}$  is similarly constructed by arg min indices. Then, by randomly selecting the same number of data samples and using the same configuration, we train two separate diffusion models on  $\mathcal{D}_{\text{farthest}}$  and  $\mathcal{D}_{\text{nearest}}$ , respectively, and then perform inference (sampling). The training loss curves and sampling results are shown in Figure 6 and Figure 7, respectively. One can observe a significant performance gap: the diffusion model trained on  $\mathcal{D}_{\text{farthest}}$  appears a higher learning loss and worse sampling quality compared to those of  $\mathcal{D}_{\text{nearest}}$ .
Figure 6: The training loss dynamics.
Figure 7: Sampling of the farthest (left) and nearest (right) clusters.
We compare the results developed in this work with former corresponding literature as follows:
- The previous work [21] also studied the adverse effect of modes shift, which particularly reported a contrastive simulation indicating the degraded performance when modeling Gaussian mixtures with the increasing distance between modes (see Figure 2 in [21] and compare with Figure 4 and Figure 5). However, the results therein are established and tested under the "pure" score matching setting, without the denoising or time-dependent dynamics. As a comparison, Theorem 2 establishes a theoretical estimate on the generalization gap for diffusion models that requires denoising, and this upper bound directly depends on the distance between modes of target distributions, instead of a circuitous characterization in [21] to attribute the difficulty of learning modes shift to increased isoperimetry of corresponding target distributions.
- Similar adverse effect of modes shift has also been theoretically analyzed and numerically verified on recurrent neural networks (RNNs), see e.g., [23, 24]. There, the modes shift is understood as a type of long-term memory. This is the phenomenon of the "curse of memory": When there is long-term memory in the target, it requires a large number of parameters for the approximation. Meanwhile, the training process will suffer from severe slowdowns. Both of these effects can be exponentially more pronounced with increasing memory or modes shift.
- The very recent work [42] also considered the problem of learning Gaussian mixtures using the denoising diffusion probabilistic model (DDPM) objective, but under a teacher-student setting. That is, given the Gaussian mixtures target, [42] parametrized the score network model in the same form of the score function target, with the goal to identify true parameters. Consequently, there are all positive convergence results developed in [42], despite that the time and sample complexity increases with the distance between modes. As a comparison, Theorem 2 adopts a pre-selected score network model without incorporating any information from the ground truth, and hence establish negative results concerning the modes shift. In fact, if the goal is to identify only true positions of the target Gaussian mixture (this is exactly the setting of [42]), the teacher-student setup seems not necessary (see Figure 5, where the true position is also learned efficiently using the one-hidden-layer Swish neural network as the score network, but the modes are always weighed incorrectly). In addition, [42] did not take the whole denoising dynamics into account. That is, the gradient descent (GD) analysis therein was performed on the denoising score matching objective successively at only two time stages: a larger  $t_1$  ("high noise") and a smaller  $t_2$  ("low noise"), which is often not the case in practice.
In this paper, we provide a theoretical analysis of the fundamental generalization aspect of training diffusion models under both the data-independent and data-dependent settings, and early-stopping estimates of the generalization gap along the training dynamics are derived. Quantitatively, the data-independent results indicate a polynomially small generalization error that escapes from the curse of dimensionality, while the data-dependent results suggest the adverse effect of modes shift in target distributions. Numerical simulations have illustrated and verified these theoretical analyses. This work forms a basic starting point for understanding the intricacies of modern deep generative modeling and corresponding central concerns such as memorization, privacy, and copyright arising from practical applications and business products. More broadly, the approach here may have the potential to be extended to other variants in the diffusion models family, including a general SDE-based design space ([20]), consistency models ([48]), rectified flows ([27, 26]), Schrödinger bridges ([59, 56, 10, 44, 47, 25]), etc. These are certainly worthy of future exploration.
We would like to thank Dr. Hongkang Yang for helpful discussions, and all the reviewers' valuable feedback and insightful suggestions to improve this work.
A Technical Results and Proofs
A.1 Data-Independent Generalization Gap
To derive the theorem for the generalization error of this score-based generative model, we first give the following lemmas.
Lemma 1 (Forward perturbation estimates). Consider the forward diffusion process with the linear drift coefficient (2). For any  $\delta >0$ ,  $\delta \ll 1$ , with the probability of at least  $1 - \delta$ , we have
$$
\left\| \boldsymbol {x} (t) \right\| _ {\infty} \lesssim C _ {T} \left(\left\| \boldsymbol {x} (0) \right\| _ {\infty} + \sqrt {\log \left(1 / \delta^ {2}\right)}\right), \tag {18}
$$
where  $C_T \coloneqq \max_{t \in [0, T]} \{r(t), r(t)v(t)\}$ .
Proof. When the drift coefficient  $\pmb{f}(\cdot, t): \mathbb{R}^d \mapsto \mathbb{R}^d$  is linear to  $\pmb{x}$ , i.e.,  $\pmb{f}(\pmb{x}, t) = f(t)\pmb{x}$ , the transition kernel  $p_{t|0}$  has a closed form (8)
$$
p _ {t \mid 0} (\boldsymbol {x} (t) | \boldsymbol {x} (0)) = \mathcal {N} (\boldsymbol {x} (t); r (t) \boldsymbol {x} (0), r ^ {2} (t) v ^ {2} (t) \boldsymbol {I} _ {d}) \tag {19}
$$
with  $r(t) \coloneqq e^{\int_0^t f(\zeta)d\zeta}, v(t) \coloneqq \sqrt{\int_0^t\frac{g^2(\zeta)}{r^2(\zeta)}d\zeta}$ . Hence, we have
$$
\boldsymbol {x} (t) = r (t) \boldsymbol {x} (0) + r (t) v (t) \boldsymbol {z}, \quad \boldsymbol {z} \sim \mathcal {N} (\boldsymbol {0}, \boldsymbol {I} _ {d}). \tag {20}
$$
For any  $\epsilon \sim \mathcal{N}(0,1), c > 1$ , we have
$$
\begin{array}{l} \mathbb {P} \{\epsilon : | \epsilon | > c \} = 2 \int_ {c} ^ {+ \infty} \frac {1}{\sqrt {2 \pi}} e ^ {- x ^ {2} / 2} d x \\ \leq \frac {1}{\sqrt {2 \pi}} \int_ {c} ^ {+ \infty} 2 x e ^ {- x ^ {2} / 2} d x = \frac {1}{\sqrt {2 \pi}} \int_ {c ^ {2}} ^ {+ \infty} e ^ {- x / 2} d x = \sqrt {\frac {2}{\pi}} e ^ {- c ^ {2} / 2}. \\ \end{array}
$$
Let  $\delta = \Theta (e^{-c^2 /2})$  , we get
$$
\mathbb {P} \{\epsilon : | \epsilon | \leq \sqrt {\log (1 / \delta^ {2})} \} \geq 1 - \delta . \tag {21}
$$
Hence, for any  $\delta \in (0,1)$  with  $\delta \ll 1$ , with the probability of at least  $1 - \delta$ , we have
$$
\left\| \boldsymbol {x} (t) \right\| _ {\infty} \lesssim C _ {T} \left(\left\| \boldsymbol {x} (0) \right\| _ {\infty} + \sqrt {\log \left(1 / \delta^ {2}\right)}\right) \tag {22}
$$
with  $C_T \coloneqq \max_{t \in [0, T]} \{r(t), r(t)v(t)\}$ . The proof is completed.
Lemma 2 (Theorem 1 in [49]). We have
$$
D _ {\mathrm {K L}} \left(p _ {0} \| p _ {0, \hat {\boldsymbol {\theta}} _ {n} (\tau)}\right) \leq \tilde {\mathcal {L}} (\hat {\boldsymbol {\theta}} _ {n} (\tau); g ^ {2} (\cdot)) + D _ {\mathrm {K L}} \left(p _ {T} \| \pi\right).
$$
Lemma 3. Both of the loss objectives  $\tilde{\mathcal{L}} (\pmb {\theta};\lambda (\cdot))$  and  $\tilde{\mathcal{L}} (\bar{\pmb{\theta}};\lambda (\cdot))$  are quadratic (and hence convex).
Proof. The convexity arises from the following points: (i) The score matching loss objectives  $\tilde{\mathcal{L}} (\pmb {\theta};\lambda (\cdot))$  (defined in (7)) and  $\bar{\tilde{\mathcal{L}}} (\bar{\theta};\lambda (\cdot))$  are  $L^2$ -metrics between the score network model and target score function; (ii) The score networks are defined as random feature models (see (12) and (13)) that are linear to trainable parameters. Therefore, using some trace techniques and basic variational calculations, it is not hard to derive the fact that the loss objectives are quadratic and hence convex with respect to trainable parameters.
(1) For  $\tilde{\mathcal{L}} (\pmb {\theta};\lambda (\cdot))$  , recall that  $s_{t,\pmb{\theta}}(\pmb {x}(t)) = \frac{1}{m}\pmb {A}\sigma (\pmb {W}\pmb {x}(t) + \pmb {U}\pmb {e}(t))$  , and let  $s_t(x(t))\coloneqq$ $\nabla_{\pmb{x}(t)}\log p_t(\pmb {x}(t)),h_1(\pmb {x},t)\coloneqq (\sqrt{\lambda(t)} /\sqrt{m})\sigma (\pmb {W}\pmb {x} + \pmb {U}\pmb {e}(t)),h_2(\pmb {x},t)\coloneqq \sqrt{\lambda(t)} s_t(\pmb {x}),$  we have
$$
\begin{array}{l} \tilde {\mathcal {L}} (\boldsymbol {\theta}; \lambda (\cdot)) = \mathbb {E} _ {t \sim \mathcal {U} (0, T)} \mathbb {E} _ {\boldsymbol {x} (t) \sim p _ {t}} \left[ h _ {1} ^ {\top} (\boldsymbol {x} (t), t) \left(\boldsymbol {A} / \sqrt {m}\right) ^ {\top} (\boldsymbol {A} / \sqrt {m}) \boldsymbol {h} _ {1} (\boldsymbol {x} (t), t) \right. \\ \left. - 2 h _ {2} ^ {\top} (\boldsymbol {x} (t), t) \left(\boldsymbol {A} / \sqrt {m}\right) h _ {1} (\boldsymbol {x} (t), t) \right] + \text {c o n s t a n t}. \\ \end{array}
$$
Since for any  $h,\bar{h},B$  ,we have
$$
\begin{array}{l} \mathbb {E} _ {t} \mathbb {E} _ {\boldsymbol {x} (t)} [ \boldsymbol {h} ^ {\top} (\boldsymbol {x} (t), t) \boldsymbol {B} \bar {\boldsymbol {h}} (\boldsymbol {x} (t), t) ] = \mathbb {E} _ {t} \mathbb {E} _ {\boldsymbol {x} (t)} [ \operatorname {t r a c e} (\boldsymbol {B} \bar {\boldsymbol {h}} (\boldsymbol {x} (t), t) \boldsymbol {h} ^ {\top} (\boldsymbol {x} (t), t)) ] \\ = \operatorname {t r a c e} \left(\boldsymbol {B} \mathbb {E} _ {t} \mathbb {E} _ {\boldsymbol {x} (t)} [ \bar {\boldsymbol {h}} (\boldsymbol {x} (t), t) \boldsymbol {h} ^ {\top} (\boldsymbol {x} (t), t) ]\right), \\ \end{array}
$$
we further get
$$
\tilde {\mathcal {L}} (\boldsymbol {\theta}; \lambda (\cdot)) = \frac {1}{m} \operatorname {t r a c e} \left(\boldsymbol {A} ^ {\top} \boldsymbol {A} \boldsymbol {B} _ {1}\right) - \frac {2}{\sqrt {m}} \operatorname {t r a c e} \left(\boldsymbol {A} \boldsymbol {B} _ {2}\right) + \text {c o n s t a n t},
$$
$$
\boldsymbol {B} _ {1} := \mathbb {E} _ {t \sim \mathcal {U} (0, T)} \mathbb {E} _ {\boldsymbol {x} (t) \sim p _ {t}} [ \boldsymbol {h} _ {1} (\boldsymbol {x} (t), t) \boldsymbol {h} _ {1} ^ {\top} (\boldsymbol {x} (t), t) ], \quad \boldsymbol {B} _ {2} := \mathbb {E} _ {t \sim \mathcal {U} (0, T)} \mathbb {E} _ {\boldsymbol {x} (t) \sim p _ {t}} [ \boldsymbol {h} _ {1} (\boldsymbol {x} (t), t) \boldsymbol {h} _ {2} ^ {\top} (\boldsymbol {x} (t), t) ].
$$
Here,  $B_{1}$  is a positive semi-definite matrix, since  $\pmb{v}^{\top}\pmb{B}_{1}\pmb{v} = \mathbb{E}_{t}\mathbb{E}_{\pmb{x}(t)}[(\pmb{v}^{\top}\pmb{h}_{1}(\pmb{x}(t),t))^{2}] \geq 0$  for any  $\pmb{v}$ . Notice that for any  $A, B$ ,
$$
\begin{array}{l} \operatorname {t r a c e} \left(\boldsymbol {A} ^ {\top} \boldsymbol {A} \boldsymbol {B}\right) = \operatorname {t r a c e} \left(\boldsymbol {A} \boldsymbol {B} \boldsymbol {A} ^ {\top}\right) = \sum_ {i, j} \boldsymbol {B} _ {i j} \left(\boldsymbol {A} _ {:, j}\right) ^ {\top} \boldsymbol {A} _ {:, i} = \operatorname {v e c} (\boldsymbol {A}) ^ {\top} (\boldsymbol {B} \otimes \boldsymbol {I}) \operatorname {v e c} (\boldsymbol {A}), \\ \operatorname {t r a c e} (\boldsymbol {A} \boldsymbol {B}) = \sum_ {j} (\boldsymbol {A} _ {:, j}) ^ {\top} (\boldsymbol {B} ^ {\top}) _ {:, j} = \operatorname {v e c} (\boldsymbol {A}) ^ {\top} \operatorname {v e c} (\boldsymbol {B} ^ {\top}), \\ \end{array}
$$
where  $\otimes$  denotes the Kronecker product. Hence
$$
\tilde {\mathcal {L}} (\boldsymbol {\theta}; \lambda (\cdot)) = \frac {1}{m} \operatorname {v e c} (\boldsymbol {A}) ^ {\top} \left(\boldsymbol {B} _ {1} \otimes \boldsymbol {I}\right) \operatorname {v e c} (\boldsymbol {A}) - \frac {2}{\sqrt {m}} \operatorname {v e c} \left(\boldsymbol {B} _ {2} ^ {\top}\right) ^ {\top} \operatorname {v e c} (\boldsymbol {A}) + \text {c o n s t a n t} \tag {23}
$$
is a quadratic function. It is straightforward to show that the eigenvalues of  $B_{1} \otimes I$  are the same as  $B_{1}$  but with multiplicity, implying that  $B_{1} \otimes I$  is also positive semi-definite. Therefore,
$$
\nabla_ {\boldsymbol {\theta}} ^ {2} \tilde {\mathcal {L}} (\boldsymbol {\theta}; \lambda (\cdot)) = \nabla_ {\operatorname {v e c} (\boldsymbol {A})} ^ {2} \tilde {\mathcal {L}} (\boldsymbol {\theta}; \lambda (\cdot)) = 2 (\boldsymbol {B} _ {1} \otimes \boldsymbol {I})
$$