SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We denote by  $\mathrm{MSE}_{\mathrm{optimal}}$  and  $\mathrm{MSE}_{\mathrm{arith-trig}}$  the estimation errors produced respectively by the optimal path and the arithmetic path with trigonometric weights. The estimation error of the optimal path can be Taylor-expanded in terms of the overlap  $(\epsilon$  and  $\epsilon^{\prime}(\pmb {x}))$  as well. We next compute the intermediate quantities that are required to obtain the estimation error.
$$
\begin{array}{l} \partial_ {t} p _ {t} ^ {\operatorname {o p t}} (\boldsymbol {x}) := \partial_ {t} \left(\left(a _ {t} \sqrt {p _ {0} (\boldsymbol {x})} + b _ {t} \sqrt {p _ {1} (\boldsymbol {x})}\right) ^ {2}\right) (143) \\ = 2 \left(\partial_ {t} a _ {t} \sqrt {p _ {0} (\boldsymbol {x})} + \partial_ {t} b _ {t} \sqrt {p _ {1} (\boldsymbol {x}))} \left(a _ {t} \sqrt {p _ {0} (\boldsymbol {x})} + b _ {t} \sqrt {p _ {1} (\boldsymbol {x})}\right) \right. (144) \\ = 2 p _ {0} (\boldsymbol {x}) \left(a _ {t} \times \partial_ {t} a _ {t}\right) + 2 p _ {1} (\boldsymbol {x}) \left(b _ {t} \times \partial_ {t} b _ {t}\right) + (145) \\ \end{array}
$$
$$
\begin{array}{l} 2 \sqrt {p _ {0} (\boldsymbol {x}) p _ {1} (\boldsymbol {x})} \left(\partial_ {t} a _ {t} \times b _ {t} + a _ {t} \times \partial_ {t} b _ {t}\right) (146) \\ = \pi \sin (\pi t) \left(\frac {p _ {1} (\boldsymbol {x}) - p _ {0} (\boldsymbol {x})}{2}\right) + \pi \cos (\pi t) \sqrt {p _ {0} (\boldsymbol {x}) p _ {1} (\boldsymbol {x})} + \epsilon \Bigg ( (147) \\ \end{array}
$$
$$
p _ {0} (\boldsymbol {x}) \left(\sin (\pi t) + \pi (t - 1) \cos (\pi t)\right) + p _ {1} (\boldsymbol {x}) \left(- \sin (\pi t) - \pi t \cos (\pi t)\right) + \tag {148}
$$
$$
\begin{array}{l} \sqrt {p _ {0} (\boldsymbol {x}) p _ {1} (\boldsymbol {x})} \left(\sin (\pi t) (1 - 2 t) \frac {\pi}{2} + \cos (\pi t) + 3\right) + o (\epsilon) (149) \\ = \pi \sin (\pi t) \left(\frac {p _ {1} (\boldsymbol {x}) - p _ {0} (\boldsymbol {x})}{2}\right) + O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right) (150) \\ = \partial_ {t} p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x}) + O \left(\epsilon^ {'} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right) (151) \\ \end{array}
$$
where  $g_{2}(\pmb{x}) = p_{0}(\pmb{x})\big(\sin (\pi t) + \pi (t - 1)\cos (\pi t)\big) + p_{1}(\pmb{x})\big(-\sin (\pi t) - \pi t\cos (\pi t)\big)$  is an integrable function. It follows,
$$
\begin{array}{l} \left(\partial_ {t} p _ {t} (\boldsymbol {x})\right) ^ {2} = \left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x}) + O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right)\right) ^ {2} (152) \\ = \left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2} + O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right) (153) \\ \end{array}
$$
by expanding and using that  $\partial_t p_t^{\mathrm{arith - trig}}(\pmb {x}) = \pi \sin (\pi t)\bigl (\frac{p_1(\pmb{x}) - p_0(\pmb{x})}{2}\bigr)$  is bounded so that  $\partial_t p_t^{\mathrm{arith - trig}}(\pmb {x})\times \epsilon^{\prime}(\pmb {x}) = O(\epsilon^{\prime}(\pmb {x}))$  . Moreover,
$$
\frac {1}{p _ {t} ^ {\operatorname {o p t}} (\boldsymbol {x})} = \frac {1}{p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x}) + O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {1} (\boldsymbol {x})\right)} = \frac {1}{p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})} \frac {1}{1 + \frac {O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {1} (\boldsymbol {x})\right)}{p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})}}. \tag {154}
$$
Denoting by  $\epsilon''(\pmb{x}) \coloneqq (O(\epsilon'(\pmb{x})) + O(\epsilon g_1(\pmb{x}))) / p_t^{\mathrm{arith-trig}}(\pmb{x})$  a third quantity which we assume goes to zero, we get
$$
\frac {1}{p _ {t} ^ {\operatorname {o p t}} (\boldsymbol {x})} = \frac {1}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} + \frac {O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right)}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})}. \tag {155}
$$
We can now write
$$
\begin{array}{l} \left(\partial_ {t} \log p _ {t} (\boldsymbol {x})\right) ^ {2} \times p _ {t} (\boldsymbol {x}) = \frac {\left(\partial_ {t} p _ {t} (\boldsymbol {x})\right) ^ {2}}{p _ {t} (\boldsymbol {x})} (156) \\ = \left(\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2} + O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right)\right) \times \left(\frac {1}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} + \frac {O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right)}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})}\right) (157) \\ = \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} + \frac {O \left(\epsilon^ {\prime} (\boldsymbol {x})\right) + O \left(\epsilon g _ {2} (\boldsymbol {x})\right)}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} + \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right) (158) \\ = \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})} + O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right) \left(1 + \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})}\right) (159) \\ \end{array}
$$
by expanding. We can then write
$$
\begin{array}{l} \mathrm {M S E} _ {\text {o p t i m a l}} (160) \\ = \frac {1}{N} \int_ {\mathbb {R} ^ {D}} \int_ {0} ^ {1} \frac {\left(\partial_ {t} p _ {t} ^ {\text {o p t}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\text {o p t}} (\boldsymbol {x})} d t d \boldsymbol {x} + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (161) \\ = \frac {1}{N} \int_ {\mathbb {R} ^ {D}} \int_ {0} ^ {1} \left(\frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})} + O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right) \left(1 + \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h - t r i g}} (\boldsymbol {x})}\right)\right) d t d \boldsymbol {x} (162) \\ + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (163) \\ = \operatorname {M S E} _ {\text {a r i t h - t r i g}} + \frac {1}{N} \int_ {\mathbb {R} ^ {D}} \int_ {0} ^ {1} O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right) \left(1 + \frac {\left(\partial_ {t} p _ {t} ^ {\text {a r i t h - t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\text {a r i t h - t r i g}} (\boldsymbol {x})}\right) d t d \boldsymbol {x} + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (164) \\ = \mathrm {M S E} _ {\text {a r i t h - t r i g}} + O \left(\frac {\epsilon^ {\prime \prime \prime}}{N}\right) + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (165) \\ \end{array}
$$
Additionally, we assume that the remainder term denoted by
$$
\epsilon^ {\prime \prime \prime} = \int_ {\mathbb {R} ^ {D}} \int_ {0} ^ {1} O \left(\epsilon^ {\prime \prime} (\boldsymbol {x})\right) \left(1 + \frac {\left(\partial_ {t} p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})\right) ^ {2}}{p _ {t} ^ {\operatorname {a r i t h} - \operatorname {t r i g}} (\boldsymbol {x})}\right) d t d \boldsymbol {x} \tag {166}
$$
is integrable and also goes to zero. On the other hand, the estimation error produced by the optimal path is known [17, Eq. 48] and the result can be Taylor-expanded as well
$$
\begin{array}{l} \mathrm {M S E} _ {\text {o p t i m a l}} = \frac {1}{N} 1 6 \alpha_ {H} ^ {2} + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (167) \\ = \frac {1}{N} 1 6 \left(\frac {\pi^ {2}}{1 6} + O (\epsilon)\right) + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) (168) \\ = \frac {1}{N} \pi^ {2} + O \left(\frac {\epsilon}{N}\right) + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right). (169) \\ \end{array}
$$
$$
\mathrm {M S E} _ {\text {a r i t h - t r i g}} = \frac {1}{N} \pi^ {2} + O \left(\frac {\epsilon}{N}\right) + o \left(\frac {1}{N}\right) + o \left(\frac {K ^ {2}}{N}\right) + O \left(\frac {\epsilon^ {\prime \prime \prime}}{N}\right). \tag {170}
$$
C Useful Lemma
Lemma 1 Chi-square divergence of between a density and a mixture We wish to upper bound the chi-square divergence between a distribution  $p(\pmb{x})$  and a mixture  $wp(\pmb{x}) + (1 - w)q(\pmb{x})$ , where  $0 < w < 1$ .
$$
\begin{array}{l} \mathcal {D} _ {\chi^ {2}} (p, w p + (1 - w) q) = \int_ {\boldsymbol {x} \in \mathbb {R} ^ {D}} \frac {p (\boldsymbol {x}) ^ {2}}{w p (\boldsymbol {x}) + (1 - w) q (\boldsymbol {x})} d \boldsymbol {x} - 1 (171) \\ = \int_ {\left\{\boldsymbol {x} \in \mathbb {R} ^ {D} \mid p (\boldsymbol {x}) <   q (\boldsymbol {x}) \right\}} \frac {p (\boldsymbol {x}) ^ {2}}{w p (\boldsymbol {x}) + (1 - w) q (\boldsymbol {x})} d \boldsymbol {x} (172) \\ + \int_ {\left\{\boldsymbol {x} \in \mathbb {R} ^ {D} \mid p (\boldsymbol {x}) > q (\boldsymbol {x}) \right\}} \frac {p (\boldsymbol {x}) ^ {2}}{w p (\boldsymbol {x}) + (1 - w) q (\boldsymbol {x})} d \boldsymbol {x} - 1 (173) \\ \leq \int_ {\left\{\boldsymbol {x} \in \mathbb {R} ^ {D} \mid p (\boldsymbol {x}) <   q (\boldsymbol {x}) \right\}} \frac {p (\boldsymbol {x}) ^ {2}}{w p (\boldsymbol {x}) + (1 - w) p (\boldsymbol {x})} d \boldsymbol {x} (174) \\ + \int_ {\left\{\boldsymbol {x} \in \mathbb {R} ^ {D} \mid p (\boldsymbol {x}) > q (\boldsymbol {x}) \right\}} \frac {p (\boldsymbol {x}) ^ {2}}{w q (\boldsymbol {x}) + (1 - w) q (\boldsymbol {x})} d \boldsymbol {x} - 1 (175) \\ \leq \int_ {\boldsymbol {x} \in \mathbb {R} ^ {D}} p (\boldsymbol {x}) d \boldsymbol {x} + \int_ {\boldsymbol {x} \in \mathbb {R} ^ {D}} \frac {p (\boldsymbol {x}) ^ {2}}{q (\boldsymbol {x})} d \boldsymbol {x} - 1 (176) \\ = \int_ {\boldsymbol {x} \in \mathbb {R} ^ {D}} \frac {p (\boldsymbol {x}) ^ {2}}{q (\boldsymbol {x})} d \boldsymbol {x} = \mathcal {D} _ {\chi^ {2}} (p, q) + 1 (177) \\ \end{array}
$$