SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {E} \left[ x ^ {2} (s) x ^ {2} (t) \right] = \frac {\sigma^ {4}}{4 a ^ {2}} \left(e ^ {2 a s} - 1\right) e ^ {2 a t} \left\{\left(e ^ {- 2 a s} - e ^ {- 2 a t}\right) + 3 \left(1 - e ^ {- 2 a s}\right) \right\}. \tag {20}
$$
Note that by symmetry, a similar expression follows for  $s \geq t$ .
Using these expressions, for the expected cost we get
$$
V _ {T} = \int_ {0} ^ {T} \mathbb {E} [ x ^ {2} (t) ] d t = \frac {\sigma^ {2}}{2 a} \int_ {0} ^ {T} \left(e ^ {2 a t} - 1\right) d t = \frac {\sigma^ {2}}{2 a} \left(\frac {e ^ {2 a T} - 1}{2 a} - T\right) \tag {21}
$$
We remark that a similar expression was previously obtained in [Bijl et al., 2016, Theorem 3]. Next, the expected estimated cost is
$$
\mathbb {E} [ \hat {V} _ {M} (h) ] = h \sum_ {k = 0} ^ {N - 1} \mathbb {E} [ x ^ {2} (k h) ] = \frac {\sigma^ {2} h}{2 a} \sum_ {k = 0} ^ {N - 1} \left(e ^ {2 a k h} - 1\right) = \frac {\sigma^ {2} h}{2 a} \left[ \frac {1 - e ^ {2 a T}}{1 - e ^ {2 a h}} - N \right]
$$
Lastly, it remains to compute the sum
$$
\begin{array}{l} \frac {h ^ {2}}{M} \sum_ {k, l = 0} ^ {N - 1} \mathbb {E} [ x ^ {2} (k h) x ^ {2} (l h) ] = \frac {2 h ^ {2}}{M} \sum_ {k <   l} ^ {N - 1} \mathbb {E} [ x ^ {2} (k h) x ^ {2} (l h) ] + \frac {h ^ {2}}{M} \sum_ {k = 0} ^ {N - 1} \mathbb {E} [ x ^ {4} (k h) ] \\ = \frac {\sigma^ {4} T \left(h ^ {2} \left(e ^ {2 a T} - 1\right) \left(8 e ^ {2 a h} + 3 e ^ {2 a T} + 1\right) + T ^ {2} \left(e ^ {2 a h} - 1\right) ^ {2} - 2 h T \left(e ^ {2 a h} - 1\right) \left(e ^ {2 a h} + 5 e ^ {2 a T}\right)\right)}{4 a ^ {2} B h \left(e ^ {2 a h} - 1\right) ^ {2}} \\ \end{array}
$$
The last equality is a cumbersome calculation that involves nested geometric sums. We verified the result using symbolic computation. For reference we provide the notebooks containing all calculations in the supplementary material. It remains to collect all terms to get the final result.
Proof of Corollary 3.2. When  $a = 0$ , the Langevin equation 7 is reduced to  $\mathrm{dx}(t) = \sigma \mathrm{dw}(t)$ . The computation of MSE can be performed similarly to that in the proof of Theorem 3.1, by using the following moment results of Wiener process.
$$
\mathbb {E} \left[ x ^ {2} (t) \right] = \sigma^ {2} t,
$$
$$
\mathbb {E} \left[ x ^ {2} (s) x ^ {2} (t) \right] = \sigma^ {4} s (t + 2 s) \quad \text {w h e n} s \leq t.
$$
It remains to compute the sums, and collect terms, as in the proof of Theorem 3.1.
It is also worth pointing out that the result of Corollary 3.2 can also be computed by taking the limit of the MSE in Theorem 3.1 when  $a \to 0$ . And the resulting MSE from the limit matches the one computed directly as in the proof above. This shows that the MSE in Theorem 3.1 is continuous at  $a = 0$ .
B.2 Undiscounted, Finite-Horizon: Approximate MSE
Proof of Corollary 3.3. For the asymptotic expansion, we use Theorem 3.1 to compute the leading terms in  $h$  of the mean-squared error:
$$
\begin{array}{l} E _ {1} (h, T, a) = \frac {\sigma^ {4} \left(e ^ {2 a T} - 1\right) ^ {2}}{1 6 a ^ {2}} h ^ {2} + \mathcal {O} \left(h ^ {3}\right), (22) \\ \frac {E _ {2} (h , T , a)}{B} = - \frac {\sigma^ {4} T \left(4 a T - e ^ {4 a T} + e ^ {2 a T} (8 a T - 4) + 5\right)}{8 a ^ {4}} \cdot \frac {1}{h B} + \frac {\sigma^ {4} T \left(1 - e ^ {4 a T} + 4 a T e ^ {2 a T}\right)}{4 a ^ {3} B} \\ - \frac {\sigma^ {4} T h (1 + 4 a T + e ^ {2 a T} (8 a T + 4) - 5 e ^ {4 a T})}{2 4 a ^ {2} B} - \frac {\sigma^ {4} T h ^ {2} \left(e ^ {4 a T} - 1\right)}{1 2 a B} + \mathcal {O} \left(h ^ {3} / B\right). (23) \\ \end{array}
$$
Next, we compute explicit upper and lower bounds on the MSE that hold for any  $h$  and  $B$ . Note that for all  $x \leq 0$ ,
$$
x ^ {2} / 4 \leq \frac {\left(- x + e ^ {x} - 1\right) ^ {2}}{\left(e ^ {x} - 1\right) ^ {2}} \leq x ^ {2}. \tag {24}
$$
Hence we can directly bound the  $E_{1}$  term from Theorem 3.1:
$$
\frac {\sigma^ {4} 4 a ^ {2} h ^ {2} \left(e ^ {2 a T} - 1\right) ^ {2}}{6 4 a ^ {4}} = \frac {\sigma^ {4} h ^ {2} \left(e ^ {2 a T} - 1\right) ^ {2}}{1 6 a ^ {2}} \leq E _ {1} \leq 4 \cdot \frac {\sigma^ {4} a ^ {2} h ^ {2} \left(e ^ {2 a T} - 1\right) ^ {2}}{1 6 a ^ {4}}
$$
To bound  $E_{2}$  note that for  $x\leq 0$
$$
\frac {1}{x ^ {2}} \leq \frac {1}{(1 - e ^ {x}) ^ {2}} \leq 1 + \frac {2}{x ^ {2}} \tag {25}
$$
Abbreviating  $E_{3} = h\left(e^{2aT} - 1\right)\left(4e^{2ah} + e^{2aT} + 1\right) - \left(e^{2ah} - 1\right)\left(e^{2ah} + 4e^{2aT} + 1\right)T$ , we have  $E_{2} = \frac{\sigma^{4}TE_{3}}{2a^{2}(1 - e^{2ah})^{2}}$ , and hence
$$
\frac {1}{8 a ^ {4} h ^ {2}} \cdot \sigma^ {4} T \cdot E _ {3} \leq E _ {2} \leq \left(1 + \frac {1}{8 a ^ {4} h ^ {2}}\right) \cdot E _ {3} \cdot \sigma^ {4} T
$$
To upper bound  $E_{2}$ , we repetitively use that for all  $x \leq 0$ ,  $1 + x \leq e^{x} \leq 1 + x + \frac{x^{2}}{2}$  and  $1 + x + \frac{x^{2}}{2} + \frac{x^{3}}{6} \leq e^{x}$ .
$$
\begin{array}{l} E _ {3} = h \left(e ^ {2 a T} - 1\right) \left(4 e ^ {2 a h} + e ^ {2 a T} + 1\right) - \left(e ^ {2 a h} - 1\right) \left(e ^ {2 a h} + 4 e ^ {2 a T} + 1\right) T \\ = 4 h e ^ {2 a T} e ^ {2 a h} + h e ^ {4 a T} - 4 h e ^ {2 a h} - h - T e ^ {4 a h} - 4 T e ^ {2 a h} e ^ {2 a T} + 4 T e ^ {2 a T} + T \\ = h \left(e ^ {2 a h} \left(4 e ^ {2 a T} - 4\right) + e ^ {4 a T} - 1\right) - 4 T e ^ {2 a T} e ^ {2 a h} - T e ^ {4 a h} + 4 T e ^ {2 a T} + T \\ \leq h \left((1 + 2 a h + 2 a ^ {2} h ^ {2} + \frac {4}{3} a ^ {3} h ^ {3}) (4 e ^ {2 a T} - 4) + e ^ {4 a T} - 1\right) \\ - 4 T e ^ {2 a T} \left(1 + 2 a h + 2 a ^ {2} h ^ {2} + \frac {4}{3} a ^ {3} h ^ {3}\right) - T \left(1 + 4 a h + 8 a ^ {2} h ^ {2} + \frac {3 2}{3} a ^ {3} h ^ {3}\right) + 4 T e ^ {2 a T} + T \\ = h \left(4 e ^ {2 a T} - 5 + e ^ {4 a T} - 8 a T e ^ {2 a T} - 4 T a\right) + h ^ {2} \left(2 a \left(4 e ^ {2 a T} - 4\right) - 8 a ^ {2} T e ^ {2 a T} - 8 a ^ {2} T\right) \\ + h ^ {3} \left(8 a ^ {2} \left(e ^ {2 a T} - 1\right) - \frac {1 6}{3} a ^ {3} T e ^ {2 a T} - \frac {3 2}{3} a ^ {3}\right) + h ^ {4} \frac {1 6}{3} \left(e ^ {2 a T} - 1\right) \\ \leq h \cdot \left(4 e ^ {2 a T} - 5 + e ^ {4 a T} - 8 a T e ^ {2 a T} - 4 T a\right) + \frac {3 2}{3} h ^ {2} a ^ {4} T ^ {3} + 1 6 h ^ {3} a ^ {4} T ^ {2} \\ \end{array}
$$
Combining the last two displays yields the claimed upper bound. The lower bound follows along the same lines. Note that the bounds can be refined by including higher-order approximations of  $e^x$ .
B.3 Undiscounted, Finite-Horizon: Optimal Step Size
Although the exact optimal step size  $h^*$  can be obtained from Theorem 3.1, such exact  $h^*$  doesn't have an explicit analytic solution in general. Numerically, we can find  $h^*$  by searching over step-sizes  $h_m = T / m$  for  $m = 1, \ldots, B$ , provided knowledge of the system parameters  $a$  and fixed horizon  $T$  or, by finding the root between 0 and 1 of the following equation
$$
\begin{array}{l} [ 5 + 4 a T - e ^ {2 a T} (4 + e ^ {2 a T} - 8 a T) ] (9 a T h + 3 T) + 2 a ^ {2} T h ^ {2} (3 7 - 5 a ^ {4 a T} + 2 8 a T + 5 6 a T e ^ {2 a T} \\ - 3 2 e ^ {2 a T}) + a ^ {2} h ^ {3} \left[ 3 B \left(e ^ {2 a T} - 1\right) ^ {2} + a T \left(9 1 - 7 e ^ {4 a T} + 6 0 a T + 1 2 0 a T e ^ {2 a T} - 8 4 e ^ {2 a T}\right) \right] = 0, \\ \end{array}
$$
where the equation is a simplified form of  $\frac{\partial}{\partial h}\mathrm{MSE}_T(h,B) = 0$
From the analysis point of view, a trivial way to see the order of  $h^*$  in terms of  $B, a, T$  is finding the dominated term by using Taylor's expansion for exponential parts (which is true for any  $h$ ) in Theorem 3.1. Such asymptotic expansion is given in Corollary 3.3. It is immediate that for  $h \geq 1$ , both Equation (22) and Equation (23) will blow up. Thus, a small  $h < 1$  is considered to minimize  $E_1(h, T, a) + \frac{E_2(h, T, a)}{B}$ . Keeping the first term in both Equation (22) and Equation (23) and solving for the optimal  $h^*$  yields the result in Equation (10).
A more precise approximation of  $h^*$  than Equation (10) is a minimizer of  $E_1(h, T, a) + \frac{E_2(h, T, a)}{B}$  truncated at  $\mathcal{O}(h^3)$ :
$$
\begin{array}{l} h ^ {*} (a, T, B) = \frac {D _ {1}}{3 D _ {3}} + \left(\frac {D _ {1} ^ {3}}{3 ^ {3} D _ {3} ^ {3}} - \frac {3 D _ {2}}{2 a ^ {2} D _ {3}} - \sqrt {\frac {9 D _ {2} ^ {2}}{4 a ^ {4} D _ {3} ^ {2}} - \frac {D _ {1} ^ {3} D _ {2}}{9 a ^ {2} D _ {3} ^ {4}}}\right) ^ {\frac {1}{3}} \\ + \left(\frac {D _ {1} ^ {3}}{3 ^ {3} D _ {3} ^ {3}} - \frac {3 D _ {2}}{2 a ^ {2} D _ {3}} + \sqrt {\frac {9 D _ {2} ^ {2}}{4 a ^ {4} D _ {3} ^ {2}} - \frac {D _ {1} ^ {3} D _ {2}}{9 a ^ {2} D _ {3} ^ {4}}}\right) ^ {\frac {1}{3}}, \tag {26} \\ \end{array}
$$
$$
D _ {1} = T \left(1 + 4 a T + e ^ {2 a T} (8 a T + 4) - 5 e ^ {4 a T}\right),
$$
$$
D _ {2} = T \left(4 a T - e ^ {4 a T} + e ^ {2 a T} (8 a T - 4) + 5\right),
$$
$$
D _ {3} = 3 B \left(e ^ {2 a T} - 1\right) ^ {2} - 4 a T \left(e ^ {4 a T} - 1\right).
$$
We can further express Equation (26) in terms of  $B$ , as
$$
\begin{array}{l} h ^ {*} (B) = \left(- \frac {T \left(4 a T - e ^ {4 a T} + e ^ {2 a T} (8 a T - 4) + 5\right)}{a ^ {2} (e ^ {2 a T} - 1) ^ {2}}\right) ^ {1 / 3} B ^ {- 1 / 3} \\ + \frac {T \left(1 + 4 a T + e ^ {2 a T} (8 a T + 4) - 5 e ^ {4 a T}\right)}{9 (e ^ {2 a T} - 1) ^ {2} B} + \frac {4 a T (e ^ {2 a T} + 1) D _ {2} ^ {1 / 3}}{9 a ^ {2 / 3} (e ^ {2 a T} - 1) ^ {5 / 3}} B ^ {- 4 / 3} \\ + \frac {4 a T ^ {2} (e ^ {2 a T} + 1) D _ {1}}{2 7 (e ^ {2 a T} - 1) ^ {3}} B ^ {- 2} + \mathcal {O} (B ^ {- 7 / 3}). \\ \end{array}
$$
where the first term is exactly the result in Equation (10).
B.4 Finite-Horizon, Discounted
As stated in Section 3.1, adding discounting in the finite-horizon setting makes the mean-squared error more involved. In the regime where  $h$  is small and  $B$  is large, a Taylor expansion characterizes the error surface as follows:
$$
\begin{array}{l} \mathrm {M S E} _ {T} (h, B, \gamma) \approx \frac {\sigma^ {4} T}{\log (\gamma) (a + \log (\gamma)) (2 a + \log (\gamma)) ^ {2}} \cdot \frac {1}{h B} + \frac {\sigma^ {4} \gamma^ {2 T} (e ^ {2 a T} - 1) ^ {2}}{1 6 a ^ {2}} \cdot h ^ {2} \\ + \frac {\sigma^ {4} \gamma^ {T} (e ^ {2 a T} - 1) (\gamma^ {T} (e ^ {2 a T} (2 a + \log (\gamma)) - \log (\gamma)) - 2 a)}{4 8 a ^ {2}} \cdot h ^ {3} + \frac {\sigma^ {4}}{1 4 4} \cdot h ^ {4} \tag {27} \\ \end{array}
$$
The approximation shows only the lowest order terms for  $1/(hB)$ ,  $\gamma^T$  and  $h$ . The derivation is given in Lemma B.1 below. The results show that main trade-off between  $h$  and  $B$  persists also for the discounted objective, as long as  $\gamma^T$  is treated as a constant relative to  $h^2$  and  $1/hB$ . In the limit where  $\gamma^T$  becomes small (e.g.,  $\gamma^T = o(h^4)$ ) the nature of the trade-off changes in that the approximation error improves to  $\mathcal{O}(h^4)$ . This can be understood from the fact that under geometric discounting combined with a decaying process, the sum of  $N = T/h$  estimation errors do not suffer a factor  $N$ , thereby removing a factor of  $1/h$  from the (non-squared) approximation error.
Lemma B.1 (Finite-horizon, discounted). In the finite-horizon with a discount factor  $\gamma \in (0,1]$  setting, the mean-squared error of the Monte-Carlo estimator is
$$
\mathrm {M S E} _ {T} (h, B, \gamma) = E _ {1} (h, T, a, \gamma) + \frac {E _ {2} (h , T , a , \gamma)}{B},
$$
$$
E _ {1} (h, T, a, \gamma) = C _ {1} (T, \gamma , a) \sigma^ {4} h ^ {2} + C _ {2} (T, \gamma , a) \sigma^ {4} h ^ {3} + \left(\frac {1}{1 4 4} + C _ {3} (T, \gamma , a)\right) \sigma^ {4} h ^ {4} + \mathcal {O} (h ^ {5}),
$$
$$
E _ {2} (h, T, a, \gamma) = \frac {\sigma^ {4} (T + \gamma^ {T} C _ {4} (T , \gamma , a))}{\log (\gamma) (a + \log (\gamma)) (2 a + \log (\gamma)) ^ {2} h} + \gamma^ {T} \mathcal {O} (1),
$$
$$
C _ {1} (T, \gamma , a) = \frac {\gamma^ {2 T} \left(e ^ {2 a T} - 1\right) ^ {2}}{1 6 a ^ {2}},
$$
$$
C _ {2} (T, \gamma , a) = \frac {\gamma^ {T} \left(e ^ {2 a T} - 1\right) \left(\gamma^ {T} \left(e ^ {2 a T} (2 a + \log (\gamma)) - \log (\gamma)\right) - 2 a\right)}{4 8 a ^ {2}},
$$
$$
C _ {3} (T, \gamma , a) = \frac {\gamma^ {T} \left[ \gamma^ {T} \left(e ^ {2 a T} (2 a + \log (\gamma)) - \log (\gamma)\right) ^ {2} - 4 a \left(e ^ {2 a T} (2 a + \log (\gamma)) - \log (\gamma)\right) \right]}{5 7 6 a ^ {2}},
$$
$C_4(T,\gamma ,a)$  is some finite constant of  $(T,\gamma ,a)$  that includes  $\gamma^T$  as a factor.
Proof. The proof follows the similar computations as those in the previous proof with a new expected cost as follows. In particular, using Lemma A.1, we get
$$
V _ {T} = \int_ {0} ^ {T} \gamma^ {t} \mathbb {E} [ x ^ {2} (t) ] d t = \frac {\sigma^ {2}}{2 a} \left(\frac {\gamma^ {T} e ^ {2 a T} - 1}{\log (\gamma) + 2 a} - \frac {\gamma^ {T} - 1}{\log (\gamma)}\right) \tag {28}
$$
Furthermore, the expected estimated cost is
$$
\mathbb {E} [ \hat {V} _ {M} (h) ] = \frac {\sigma^ {2} h}{2 a} \sum_ {k = 0} ^ {N - 1} \gamma^ {k h} \left(e ^ {2 a k h} - 1\right) = \frac {\sigma^ {2} h}{2 a} \left(\frac {1 - \gamma^ {T} e ^ {2 a T}}{1 - \gamma^ {h} e ^ {2 a h}} - \frac {1 - \gamma^ {T}}{1 - \gamma^ {h}}\right).
$$
Finally, the sum containing the fourth order cross-moments is
$$
\frac {h ^ {2}}{M} \sum_ {k, l = 0} ^ {N - 1} \gamma^ {k h + l h} \mathbb {E} [ x ^ {2} (k h) x ^ {2} (l h) ] = \frac {2 h ^ {2}}{M} \sum_ {k <   l} ^ {N - 1} \gamma^ {k h + l h} \mathbb {E} [ x ^ {2} (k h) x ^ {2} (l h) ] + \frac {h ^ {2}}{M} \sum_ {k = 0} ^ {N - 1} \gamma^ {2 k h} \mathbb {E} [ x ^ {4} (k h) ].
$$
While not impossible to calculate on paper, a written derivation is beyond the scope of this work. Instead, we rely on symbolic computation to obtain the expression and corresponding Taylor approximations. The notebooks containing all derivations are provided in the supplementary material.
B.5 Infinite Horizon: Proof of Theorem 3.6
Proof. The proof relies on the decomposition provided in Equation (12). It only remains to compute the following cross term.
$$
\begin{array}{l} \mathbb {E} \left[ \hat {V} _ {M} (h) - V _ {T} \right] V _ {T, \infty} \\ = \frac {\sigma^ {4}}{2 a} \left(\frac {\gamma^ {T}}{\log (\gamma)} - \frac {\gamma^ {T} e ^ {2 a T}}{\log (\gamma) + 2 a}\right) \left[ \frac {h}{2 a} \left(\frac {1 - \gamma^ {T} e ^ {2 a T}}{1 - \gamma^ {h} e ^ {2 a h}} - \frac {1 - \gamma^ {T}}{1 - \gamma^ {h}}\right) - \frac {1}{2 a} \left(\frac {\gamma^ {T} e ^ {2 a T} - 1}{\log (\gamma) + 2 a} - \frac {\gamma^ {T} - 1}{\log (\gamma)}\right) \right] \\ = \frac {\sigma^ {4} \gamma^ {2 T} (e ^ {2 a T} - 1) (\log (\gamma) (e ^ {2 a T} - 1) - 2 a)}{8 a ^ {2} \log (\gamma) (2 a + \log (\gamma))} h + \\ \frac {\sigma^ {4} \gamma^ {T} \left(2 a + \log (\gamma) - e ^ {2 a T} \log (\gamma)\right) \left(2 a \left(\gamma^ {T} e ^ {2 a T} - 1\right) + \gamma^ {T} \log (\gamma) \left(e ^ {2 a T} - 1\right)\right)}{4 8 a ^ {2} \log (\gamma) \left(2 a + \log (\gamma)\right)} h ^ {2} + \mathcal {O} (h ^ {3}). \\ \end{array}
$$
Thus, the mean-squared error  $\mathrm{MSE}_{\infty}(h,B,T,\gamma) = \mathbb{E}\big[(\hat{V}_M(h) - V_\infty)^2\big]$  is obtained by combining the above computation with Equation (11) and Lemma B.1.
It is worth pointing out the trade-off always exists with or without the assumption  $\gamma^T = o(h^4)$  from  $\mathrm{MSE}_{\infty}(h,B,T,\gamma)$ . For example, if  $\gamma^T$  is constant with respect to  $h$ ,
$$
\begin{array}{l} \mathrm {M S E} _ {\infty} (h, B, T, \gamma) = \sigma^ {4} T \frac {1}{\log (\gamma) (a + \log (\gamma)) (2 a + \log (\gamma)) ^ {2}} \cdot \frac {1}{h B} \\ + \frac {\sigma^ {4} \gamma^ {2 T} \left(e ^ {2 a T} - 1\right) \left(\log (\gamma) \left(e ^ {2 a T} - 1\right) - 2 a\right)}{8 a ^ {2} \log (\gamma) \left(2 a + \log (\gamma)\right)} h \\ + \frac {\sigma^ {4} \gamma^ {2 T} [ (1 - e ^ {2 a T}) \log (\gamma) + 2 a ] ^ {2}}{4 a ^ {2} [ \log (\gamma) (a + \log (\gamma)) ] ^ {2}} + \mathcal {O} (h ^ {2}) + \mathcal {O} (B ^ {- 1}). \\ \end{array}
$$
From the above expression,  $\mathrm{MSE}_{\infty}(h,B,T,\gamma)$  contains a constant term in this case. For the other cases of  $\gamma^T$ ,  $\mathrm{MSE}_{\infty}(h,B,T,\gamma)$  can be obtained similarly by combining the cross term in the proof of Theorem 3.6, Equation (11) and Lemma B.1. The case where  $\gamma^T = o(h^4)$  is particularly interesting because even as  $\gamma^T\to 0$  at such a fast rate, there is still a trade-off that never vanishes.
C Vector Case Analysis
C.1 Finite-Horizon, Undiscounted: Proof of Theorem 3.4
Proof. Consider the n-dimensional system that the solution of the trajectory of  $X(t)$  is
$$
X (t) = \sigma \int_ {0} ^ {t} e ^ {A (t - s)} d W (t).
$$
Since  $A$  is a diagonalizable matrix, we can decompose  $A$  as  $A = P^{-1}DP$ , where  $P$  is a invertible matrix (not necessarily to be orthogonal) and  $D$  is a diagonal matrix whose diagonal entries  $(\lambda_1, \dots, \lambda_n)$  are corresponding to the eigenvalues of the matrix  $A$ . Followed by which, we can decompose the matrix exponential of  $A$  as:
$$
e ^ {A t} = P ^ {- 1} e ^ {D t} P.
$$
Define the "diagonalized" process  $\tilde{X} (\cdot)$  as:
$$
\begin{array}{l} P X (t) = P \sigma \int_ {0} ^ {t} e ^ {A (t - s)} \mathrm {d} W (s) \\ = \sigma P P ^ {- 1} \int_ {0} ^ {t} e ^ {D (t - s)} P \mathrm {d} W (s) \\ = \sigma \int_ {0} ^ {t} e ^ {D (t - s)} \mathrm {d} \tilde {W} (s) =: \tilde {X} (t) \\ \end{array}
$$
where  $\tilde{W}(s)$  is a Wiener process (with dependent components when  $P$  is not orthogonal). This implies that  $X(\cdot) = P^{-1}\tilde{X}(\cdot)$ .
To see  $\tilde{X}_i(t)$  clearly, we denote  $P = [p_{ij}]_{i,j=1}^n$ , and  $\tilde{X}_i(t) = (\phi_1^{(i)}(t), \dots, \phi_n^{(i)}(t))^{\top}$ , then  $\phi_l^{(i)}(t) = \sum_{j=1}^n p_{lj} \sigma \int_0^t e^{\lambda_l(t - s)} \, \mathrm{d}w_j^{(i)}(s)$  for each  $l \in \{1, \dots, n\}$ . Particularly, in such an expression,  $w_j^{(i)}(s)$  are independent Wiener processes for different  $i$  or  $j$ . Correspondingly,  $\tilde{X}(t) = (\phi_1(t), \dots, \phi_n(t))^{\top}$ , and  $\phi_l(t) = \sum_{j=1}^n p_{lj} \sigma \int_0^t e^{\lambda_l(t - s)} \, \mathrm{d}w_j(s)$  for each  $l \in \{1, \dots, n\}$ , where  $w_j(s)$  are independent Wiener processes for different  $j$ .
By trace operation, we can rewrite  $\hat{V}_M(h)$  as follows:
$$
\begin{array}{l} \hat {V} _ {M} (h) = \frac {1}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} h X \left(t _ {k}\right) ^ {\top} Q X \left(t _ {k}\right) \\ = \operatorname {t r} \left\{\frac {1}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} h \tilde {X} (t _ {k}) ^ {\top} P ^ {- \top} Q P ^ {- 1} \tilde {X} (t _ {k}) \right\} \\ = \mathrm {t r} \left\{P ^ {- \top} Q P ^ {- 1} \hat {\mathcal {V}} _ {M} (h) \right\}, \\ \end{array}
$$
where  $\hat{\mathcal{V}}_M(h) = \frac{1}{M}\sum_{i=1}^{M}\sum_{k=0}^{N-1}h\tilde{X}(t_k)\tilde{X}(t_k)^\top \in \mathbb{R}^{n\times n}$ .
Similarly,  $V_{T} = \mathrm{tr}\left\{P^{-\top}QP^{-1}\mathcal{V}_{T}\right\}$ , where  $\mathcal{V}_T = \int_0^T\mathbb{E}[\tilde{X} (t)\tilde{X} (t)^\top ]\mathrm{d}t$
Therefore, the  $\mathrm{MSE}_T(h,B)$  can be written as
$$
\mathbf {M S E} _ {T} (h, B) = \mathbb {E} \left[ \left(\hat {V} _ {M} (h) - V _ {T}\right) ^ {2} \right] = \mathbb {E} \left[ \operatorname {t r} \left\{P ^ {- \top} Q P ^ {- 1} \left(\hat {\mathcal {V}} _ {M} (h) - \mathcal {V} _ {T}\right) \right\} ^ {2} \right]. \tag {29}
$$
For notional simplicity, we denote matrix  $P^{-\top}QP^{-1} \eqqcolon B = [b_{lj}]_{l,j=1}^{n}$  and  $\hat{\mathcal{V}}_M(h) - \mathcal{V}_T \eqqcolon C = [c_{lj}]_{l,j=1}^n$ .
Noting the fact that
$$
\mathrm {M S E} _ {T} (h, B) = \mathbb {E} \left[ \left(\sum_ {l, j} b _ {j l} c _ {l j}\right) ^ {2} \right] = \sum_ {l _ {1}, j _ {1}, l _ {2}, j _ {2}} b _ {j _ {1} l _ {1}} b _ {j _ {2} l _ {2}} \mathbb {E} \left[ c _ {l _ {1} j _ {1}} c _ {l _ {2} j _ {2}} \right], \tag {30}
$$
it is sufficient to find  $\mathrm{MSE}_T$  by only computing  $\mathbb{E}\left[c_{l_1j_1}c_{i_2j_2}\right]$ .
We first introduce the following expectations that are used in the computations. For any  $s \leq t$
$$
\mathbb {E} \left[ \int_ {0} ^ {t} e ^ {\lambda_ {1} (t - u)} \mathrm {d} w (u) \int_ {0} ^ {s} e ^ {\lambda_ {2} (s - u)} \mathrm {d} w (u) \right] = \frac {e ^ {\lambda_ {1} t + \lambda_ {2} s}}{\lambda_ {1} + \lambda_ {2}} \left(1 - e ^ {- (\lambda_ {1} + \lambda_ {2}) s}\right), \tag {31}
$$
$$
\begin{array}{l} \mathbb {E} \left[ \int_ {0} ^ {s} e ^ {\lambda_ {1} (s - u)} \mathrm {d} w (u) \int_ {0} ^ {s} e ^ {\lambda_ {2} (s - u)} \mathrm {d} w (u) \int_ {0} ^ {t} e ^ {\lambda_ {3} (t - u)} \mathrm {d} w (u) \int_ {0} ^ {t} e ^ {\lambda_ {4} (t - u)} \mathrm {d} w (u) \right] \\ = e ^ {(\lambda_ {1} + \lambda_ {2}) s + (\lambda_ {3} + \lambda_ {4}) t} \left[ \frac {1}{(\lambda_ {1} + \lambda_ {2}) (\lambda_ {3} + \lambda_ {4})} \left(1 - e ^ {- (\lambda_ {1} + \lambda_ {2}) s}\right) \left(1 - e ^ {- (\lambda_ {3} + \lambda_ {4}) s}\right) \right. \\ + \frac {1}{\left(\lambda_ {1} + \lambda_ {3}\right) \left(\lambda_ {2} + \lambda_ {4}\right)} \left(1 - e ^ {- \left(\lambda_ {1} + \lambda_ {3}\right) s}\right) \left(1 - e ^ {- \left(\lambda_ {2} + \lambda_ {4}\right) s}\right) \\ + \frac {1}{\left(\lambda_ {1} + \lambda_ {4}\right) \left(\lambda_ {2} + \lambda_ {3}\right)} \left(1 - e ^ {- \left(\lambda_ {1} + \lambda_ {4}\right) s}\right) \left(1 - e ^ {- \left(\lambda_ {2} + \lambda_ {3}\right) s}\right) \\ \left. + \frac {1}{\left(\lambda_ {1} + \lambda_ {2}\right) \left(\lambda_ {3} + \lambda_ {4}\right)} \left(1 - e ^ {- \left(\lambda_ {1} + \lambda_ {2}\right) s}\right) \left(e ^ {- \left(\lambda_ {3} + \lambda_ {4}\right) s} - e ^ {- \left(\lambda_ {3} + \lambda_ {4}\right) t}\right) \right] \tag {32} \\ \end{array}
$$
$$
\int_ {0} ^ {T} \mathbb {E} \left[ \int_ {0} ^ {t} e ^ {\lambda_ {1} (t - u)} \mathrm {d} w (u) \int_ {0} ^ {s} e ^ {\lambda_ {2} (- u)} \mathrm {d} w (u) \right] \mathrm {d} t = \frac {e ^ {(\lambda_ {1} + \lambda_ {2}) T} - 1 - (\lambda_ {1} + \lambda_ {2}) T}{(\lambda_ {1} + \lambda_ {2}) ^ {2}}. \tag {33}
$$
By using the definitions of  $\hat{\mathcal{V}}_M(h)$  and  $\mathcal{V}_T$ , it is trivial to see for any  $l, j \in \{1, \dots, n\}$
$$
\begin{array}{l} c _ {l j} = \frac {1}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} h \phi_ {l} ^ {(i)} (k h) \phi_ {j} ^ {(i)} (k h) - \int_ {0} ^ {T} \mathbb {E} [ \phi_ {l} (t) \phi_ {j} (t) ] d t \\ = \frac {h \sigma^ {2}}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} \left(\sum_ {\alpha = 1} ^ {n} p _ {l \alpha} \int_ {0} ^ {k h} e ^ {\lambda_ {l} (k h - s)} d w _ {\alpha} ^ {(i)} (s)\right) \left(\sum_ {\alpha = 1} ^ {n} p _ {j \alpha} \int_ {0} ^ {k h} e ^ {\lambda_ {j} (k h - s)} d w _ {\alpha} ^ {(i)} (s)\right) \\ - \sigma^ {2} \int_ {0} ^ {T} \mathbb {E} \left[ \left(\sum_ {\alpha = 1} ^ {n} p _ {l \alpha} \int_ {0} ^ {t} e ^ {\lambda_ {l} (t - s)} d w _ {\alpha} (s)\right) \left(\sum_ {\alpha = 1} ^ {n} p _ {j \alpha} \int_ {0} ^ {t} e ^ {\lambda_ {j} (t - s)} d w _ {\alpha} (s)\right) \right] d t \\ = \sum_ {\alpha = 1} ^ {n} p _ {l \alpha} p _ {j \alpha} \left[ \frac {h \sigma^ {2}}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} \left(\int_ {0} ^ {k h} e ^ {\lambda_ {l} (k h - s)} d w _ {\alpha} ^ {(i)} (s)\right) \left(\int_ {0} ^ {k h} e ^ {\lambda_ {j} (k h - s)} d w _ {\alpha} ^ {(i)} (s)\right) \right. \\ - \sigma^ {2} \int_ {0} ^ {T} \mathbb {E} \left[ \left(\int_ {0} ^ {t} e ^ {\lambda_ {l} (t - s)} \mathrm {d} w _ {\alpha} (s)\right) \left(\int_ {0} ^ {t} e ^ {\lambda_ {j} (t - s)} \mathrm {d} w _ {\alpha} (s)\right) \right] \mathrm {d} t \biggr ] + \\ \sum_ {\alpha \neq \beta} p _ {l \alpha} p _ {j \beta} \left[ \frac {h \sigma^ {2}}{M} \sum_ {i = 1} ^ {M} \sum_ {k = 0} ^ {N - 1} \left(\int_ {0} ^ {k h} e ^ {\lambda_ {l} (k h - s)} \mathrm {d} w _ {\alpha} ^ {(i)} (s)\right) \left(\int_ {0} ^ {k h} e ^ {\lambda_ {j} (k h - s)} \mathrm {d} w _ {\beta} ^ {(i)} (s)\right) \right], \\ \end{array}
$$
where the last equation is due to the fact that for  $\alpha \neq \beta$
$$
\mathbb {E} \left[ \left(\int_ {0} ^ {t} e ^ {\lambda_ {l} (t - s)} \mathrm {d} w _ {\alpha} (s)\right) \left(\int_ {0} ^ {t} e ^ {\lambda_ {j} (t - s)} \mathrm {d} w _ {\beta} (s)\right) \right] = 0.
$$
Thus, for any  $l_1, l_2, j_1, j_2 \in \{1, \dots, n\}$ ,