SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \max  _ {t = s + \rho + 1} ^ {e - \rho} | | \widetilde {F} _ {t, h} ^ {s, e} (x) - \widetilde {f} _ {t} ^ {s, e} (x) | | _ {L _ {2}} \leq \max  _ {t = s + \rho + 1} ^ {e - \rho} \widetilde {C} _ {\mathcal {X}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \widetilde {F} _ {t, h} ^ {s, e} (x) - \widetilde {f} _ {t} ^ {s, e} (x) \right| \\ \leq 2 \widetilde {C} _ {\mathcal {X}} C \sqrt {\frac {\log T}{h ^ {p}}} + \frac {2 \widetilde {C} _ {\mathcal {X}} C _ {1} \sqrt {p}}{h ^ {p}} + 2 \widetilde {C} _ {\mathcal {X}} C _ {2} \sqrt {T} h ^ {r} \\ \end{array}
$$
where  $\widetilde{C}_{\mathcal{X}}$  is the volume of the set  $\mathcal{X}$ . Moreover, using inequality (47), we have that
$$
\begin{array}{l} \operatorname * {m a x} _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (\cdot - X _ {t}) - \int \mathcal {K} _ {h} (\cdot - z) d F _ {t} (z)\right) \right| \right| _ {L _ {2}} \\ \leq C _ {\mathcal {X}} \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ = O _ {p} \left(\sqrt {\frac {\log T}{h ^ {p}}}\right). \\ \end{array}
$$
H  $\alpha$ -mixing condition
A process  $(X_{t},t\in \mathbb{Z})$  is said to be  $\alpha$ -mixing if
$$
\alpha_ {k} = \sup  _ {t \in \mathbb {Z}} \alpha (\sigma (X _ {s}, s \leq t), \sigma (X _ {s}, s \geq t + k)) \longrightarrow_ {k \rightarrow \infty} 0.
$$
The strong mixing, or  $\alpha$ -mixing coefficient between two  $\sigma$ -fields  $\mathcal{A}$  and  $\mathcal{B}$  is defined as
$$
\alpha (\mathcal {A}, \mathcal {B}) = \sup  _ {A \in \mathcal {A}, B \in \mathcal {B}} | \mathbb {P} (A \cap B) - \mathbb {P} (A) \mathbb {P} (B) |.
$$
Suppose  $X$  and  $Y$  are two random variables. Then for positive numbers  $p^{-1} + q^{-1} + r^{-1} = 1$ , it holds that
$$
| \operatorname {C o v} (X, Y) | \leq 4 \| X \| _ {L _ {p}} \| Y \| _ {L _ {q}} \left\{\alpha (\sigma (X), \sigma (Y)) \right\} ^ {1 / r}.
$$
Let  $\left\{Z_t\right\}_{t = -\infty}^{\infty}$  be a stationary time series vectors. Denote the alpha mixing coefficients of  $k$  to be
$$
\alpha (k) = \alpha \Big (\sigma \Big \{\dots , Z _ {t - 1}, Z _ {t} \Big \}, \sigma \Big \{Z _ {t + k}, Z _ {t + k + 1}, \dots \Big \} \Big).
$$
Note that the definition is independent of  $t$ .
H.1 Maximal Inequality
The unstationary version of the following lemma is in Lemma B.5. of Kirch (2006).
Lemma 1. Suppose  $\left\{y_{i}\right\}_{i = 1}^{\infty}$  is a stationary alpha-mixing time series with mixing coefficient  $\alpha (k)$  and that  $\mathbb{E}\big(y_i\big) = 0$ . Suppose that there exists  $\delta, \Delta > 0$  such that
$$
\mathbb {E} \left(\left| y _ {i} \right| ^ {2 + \delta + \Delta}\right) \leq D _ {1}
$$
$$
\sum_ {k = 0} ^ {\infty} (k + 1) ^ {\delta / 2} \alpha (k) ^ {\Delta / (2 + \delta + \Delta)} \leq D _ {2}.
$$
$$
\mathbb {E} \Big (\max  _ {k = 1, \dots , n} \left| \sum_ {i = 1} ^ {k} y _ {i} \right| ^ {2 + \delta} \Big) \leq D n ^ {(2 + \delta) / 2},
$$
where  $D$  only depends on  $\delta$  and the joint distribution of  $\left\{y_{i}\right\}_{i = 1}^{\infty}$ .
Proof. This is Lemma B.8. of Kirch (2006).
Lemma 2. Suppose that there exists  $\delta, \Delta > 0$  such that
Then it holds that for any  $d > 0, 0 < \nu < 1$  and  $x > 0$ ,
$$
\mathbb {P} \Big (\max  _ {k \in [ \nu d, d ]} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {k}} \geq x \Big) \leq C x ^ {- 2 - \delta},
$$
where  $C$  is some constant.
$$
S _ {d} ^ {*} = \max  _ {k = 1, \dots , d} \left| \sum_ {i = 1} ^ {k} y _ {i} \right|.
$$
Then Lemma 1 implies that
$$
\left\| S _ {d} ^ {*} \right\| _ {L _ {2 + \delta}} \leq C _ {1} d ^ {1 / 2}
$$
Therefore it holds that
$$
\mathbb {P} \Big (\Big | \frac {S _ {d} ^ {*}}{\sqrt {d}} \Big | \geq x \Big) = \mathbb {P} \Big (\Big | \frac {S _ {d} ^ {*}}{\sqrt {d}} \Big | ^ {2 + \delta} \geq x ^ {2 + \delta} \Big) \leq C _ {1} x ^ {- 2 - \delta}.
$$
$$
\frac {\left| S _ {d} ^ {*} \right|}{\sqrt {d}} = \max _ {k = 1, \dots , d} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {d}} \geq \max _ {k \in [ \nu d, d ]} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {d}} \geq \max _ {k \in [ \nu d, d ]} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {k / \nu}}
$$
$$
\mathbb {P} \Big (\max  _ {k \in [ \nu d, d ]} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {k}} \geq x / \sqrt {\nu} \Big) \leq \mathbb {P} \Big (\left| \frac {S _ {d} ^ {*}}{\sqrt {d}} \right| \geq x \Big) \leq C _ {1} x ^ {- 2 - \delta},
$$
$$
\mathbb {P} \Big (\max  _ {k \in [ \nu d, d ]} \frac {\left| \sum_ {i = 1} ^ {k} y _ {i} \right|}{\sqrt {k}} \geq x \Big) \leq C _ {2} x ^ {- 2 - \delta}.
$$
Lemma 3. Let  $\nu >0$  be given. Under the same assumptions as in Lemma 1, for any  $0 < a < 1$  it holds that
$$
\mathbb {P} \left(\left| \sum_ {i = 1} ^ {r} y _ {i} \right| \leq \frac {C}{a} \sqrt {r} \left\{\log (r \nu) + 1 \right\} f o r a l l r \geq 1 / \nu\right) \geq 1 - a ^ {2},
$$
where  $C$  is some absolute constant.
Proof. Let  $s \in \mathbb{Z}^+$  and  $\mathcal{T}_s = \left[2^s / \nu, 2^{s + 1} / \nu\right]$ . By Lemma 3, for all  $x \geq 1$ ,
$$
\mathbb {P} \Big (\sup _ {r \in \mathcal {T} _ {s}} \frac {\left| \sum_ {i = 1} ^ {r} y _ {i} \right|}{\sqrt {r}} \geq x \Big) \leq C _ {1} x ^ {- 2 - \delta} \leq C _ {1} x ^ {- 2}.
$$
Therefore by a union bound, for any  $0 < a < 1$ ,
$$
\mathbb {P} \Big (\exists s \in \mathbb {Z} ^ {+}: \sup  _ {r \in \mathcal {T} _ {s}} \frac {\left| \sum_ {i = 1} ^ {r} y _ {i} \right|}{\sqrt {r}} \geq \frac {\sqrt {C _ {1}}}{a} (s + 1) \Big) \leq \sum_ {s = 0} ^ {\infty} \frac {a ^ {2}}{(s + 1) ^ {2}} = a ^ {2} \pi^ {2} / 6.
$$
For any  $r \in \left[2^s / \nu, 2^{s+1} / \nu\right]$ ,  $s \leq \log(r\nu) / \log(2)$ , and therefore
$$
\mathbb {P} \Big (\exists s \in \mathbb {Z} ^ {+}: \sup  _ {r \in \mathcal {T} _ {s}} \frac {\left| \sum_ {i = 1} ^ {r} y _ {i} \right|}{\sqrt {r}} \geq \frac {\sqrt {C _ {1}}}{a} \Big \{\frac {\log (r \nu)}{\log (2)} + 1 \Big \} \Big) \leq a ^ {2} \pi^ {2} / 6.
$$
Equation (2) directly gives
$$
\mathbb {P} \Big (\sup  _ {r \in \mathcal {T} _ {s}} \frac {\left| \sum_ {i = 1} ^ {r} y _ {i} \right|}{\sqrt {r}} \geq \frac {C}{a} \left\{\log (r \nu) + 1 \right\} \Big) \leq a ^ {2}.
$$
H.2 Central Limit theorem
Below is the central limit theorem for  $\alpha$ -mixing random variable. We refer to Doukhan (1994) for more details.
Lemma 4. Let  $\left\{Z_{t}\right\}$  be a centred  $\alpha$ -mixing stationary time series. Suppose for the mixing coefficients and moments, for some  $\delta >0$  it holds
$$
\sum_ {k = 1} ^ {\infty} \alpha_ {k} ^ {\delta / (2 + \delta)} <   \infty , \quad \mathbb {E} \Big [ \Big | Z _ {1} \Big | ^ {2 + \delta} <   \infty \Big ].
$$
Denote  $S_{n} = \sum_{t = 1}^{n}Z_{t}$  and  $\sigma_n^2 = \mathbb{E}\Big[\big|S_n\big|^2\Big]$ . Then
$$
\frac {S _ {\lfloor n t \rfloor}}{\sigma_ {n}} \to W (t),
$$
where convergence is in Skorohod topology and  $W(t)$  is the standard Brownian motion on  $[0,1]$ .
I Additional Technical Results
Lemma 5. Let  $\mathcal{I}$  be defined as in Definition 2 and suppose Assumption 1 e holds. Then for each change point  $\eta_{k}$  there exists a seeded interval  $\mathcal{I}_k = (s_k,e_k]$  such that
a.  $\mathcal{I}_k$  contains exactly one change point  $\eta_k$ ;
b.  $\min \{\eta_k - s_k, e_k - \eta_k\} \geq \frac{1}{16}\Delta$ ; and
c.  $\max \{\eta_k - s_k, e_k - \eta_k\} \leq \frac{9}{10}\Delta$
Proof. These are the desired properties of seeded intervals by construction. The proof is the same as theorem 3 of Kovacs et al. (2020) and is provided here for completeness.
Let  $k \in \{1, \dots, \mathfrak{K}\}$ , where  $\mathfrak{K}$  is from definition 2. By construction of seeded intervals, we have that in  $\mathcal{I}_k$ , we can find an interval  $(\lfloor (i - 1)T2^{-k} \rfloor, \lceil (i - 1)T2^{-k} + T2^{-k + 1} \rceil]$ , for some  $i \in 1, \dots, 2^k - 1$ , such that
$$
\min  \left\{\eta_ {k} - \lfloor (i - 1) T 2 ^ {- k} \rfloor , \lceil (i - 1) T 2 ^ {- k} + T 2 ^ {- k + 1} \rceil - \eta_ {k} \right\} \geq \frac {l _ {k}}{4} \tag {50}
$$
$$
\left. \max  \left\{\eta_ {k} - \lfloor (i - 1) T 2 ^ {- k} \right\rfloor , \lceil (i - 1) T 2 ^ {- k} + T 2 ^ {- k + 1} \rceil - \eta_ {k} \right\} \leq l _ {k}, \tag {51}
$$
where  $l_{k} = T2^{-k + 1}$ , is the size of each interval in  $\mathcal{I}_k$ . By the choice of  $\Re$ , there exist  $k$ , such that  $l_{k} = \frac{9}{10}\Delta$ , from where the claim is followed.
Lemma 6. Let  $\{X_i\}_{i=1}^T$  be random grid points sampled from a common density function  $f_t: \mathbb{R}^p \to \mathbb{R}$ , satisfying Assumption 1-a and -b. Under Assumption (2), the density estimator of the sampling distribution  $\mu$ ,
$$
\widehat {f} _ {t} (x) = \frac {1}{T} \sum_ {t = 1} ^ {T} \mathcal {K} _ {h} (x - X _ {i}), \quad x \in \mathbb {R} ^ {p},
$$
$$
\left| \left| \widehat {f} _ {T} - f _ {t} \right| \right| _ {L _ {\infty}} = O _ {p} \left(\left(\frac {\log (T)}{T}\right) ^ {\frac {2 r}{2 r + p}}\right). \tag {52}
$$
The verification of these bounds can be found in many places in the literature. See for example Yu (1993) and Tsybakov (2009).
Remark 2. Even more, by Assumption 1,
$$
\left\| \widehat {f} _ {T} - f _ {t} \right\| _ {L _ {2}} \leq C _ {\mathcal {X}} \| \widehat {f} _ {T} - f _ {t} \| _ {L _ {\infty}} = O \left(\left(\frac {\log (T)}{T}\right) ^ {\frac {2 r}{2 r + p}}\right) \tag {53}
$$
with high probability. Therefore, given that
$$
\kappa = \frac {\left|\left| \sqrt {\frac {\eta_ {k + 1} - \eta_ {k}}{\left(\eta_ {k + 1} - \eta_ {k - 1}\right)\left(\eta_ {k} - \eta_ {k - 1}\right)}} \sum_ {i = \eta_ {k - 1} + 1} ^ {\eta_ {k}} f _ {i} - \sqrt {\frac {\left(\eta_ {k} - \eta_ {k - 1}\right)}{\left(\eta_ {k + 1} - \eta_ {k - 1}\right)\left(\eta_ {k + 1} - \eta_ {k}\right)}} \right| \sum_ {i = \eta_ {k} + 1} ^ {\eta_ {k + 1}} f _ {i} \right|\left. \right| L _ {2}}{\sqrt {\frac {\left(\eta_ {k} - \eta_ {k - 1}\right)\left(\eta_ {k + 1} - \eta_ {k}\right)}{\eta_ {k + 1} - \eta_ {k - 1}}}} \tag {54}
$$
and (5), by triangle inequality, (53), and Lemma 5, we have that
$$
| \kappa - \widehat {\kappa} | = O _ {p} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right).
$$
From here, and Assumption 2, if  $h_1 = O(\kappa^{\frac{1}{r}})$  and  $h_2 = O(\widehat{\kappa}^{\frac{1}{r}})$ , we conclude that
$$
| | F _ {t, h _ {1}} - F _ {t, h _ {2}} | | _ {L _ {2}} ^ {2} = O \Big (\frac {| \kappa - \widehat {\kappa} |}{\kappa^ {\frac {p}{r} + 1}} \Big).
$$
$$
\begin{array}{l} \left\| F _ {t, h _ {1}} - F _ {t, h _ {2}} \right\| _ {L _ {2}} ^ {2} \\ = \int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right) - \frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right)\right) ^ {2} d x \\ = \int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {1} ^ {p}} \mathcal {K} (\frac {x - X _ {t}}{h _ {1}})\right) ^ {2} - 2 \frac {1}{h _ {1} ^ {p}} \mathcal {K} (\frac {x - X _ {t}}{h _ {1}}) \frac {1}{h _ {2} ^ {p}} \mathcal {K} (\frac {x - X _ {t}}{h _ {2}}) + \left(\frac {1}{h _ {2} ^ {p}} \mathcal {K} (\frac {x - X _ {t}}{h _ {2}})\right) ^ {2} d x. \\ \end{array}
$$
Now, we analyze the two following terms,
$$
I _ {1} = \int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right)\right) ^ {2} - \frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right) \frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right) d x
$$
$$
I _ {2} = \int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right)\right) ^ {2} - \frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right) \frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right) d x.
$$
For  $I_{1}$ , letting  $u = \frac{x - X_t}{h_1}$ , we have that
$$
\int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right)\right) ^ {2} d x = \int_ {\mathbb {R} ^ {p}} \frac {1}{h _ {1} ^ {p}} \left(\mathcal {K} (u)\right) ^ {2} d u
$$
and, letting  $v = \frac{x - X_t}{h_2}$ , we have that
$$
\int_ {\mathbb {R} ^ {p}} \frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right) \frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right) d x = \int_ {\mathbb {R} ^ {p}} \frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(v \frac {h _ {2}}{h _ {1}}\right) \mathcal {K} (v) d v.
$$
Therefore, by Assumption 2 and the Mean Value Theorem,
$$
\begin{array}{l} I _ {1} = \frac {1}{h _ {1} ^ {p}} \int_ {\mathbb {R} ^ {p}} \mathcal {K} (v) \left(\mathcal {K} (v) - \mathcal {K} \left(v \frac {h _ {2}}{h _ {1}}\right)\right) d v \leq C \frac {1}{h _ {1} ^ {p}} \left| 1 - \frac {h _ {2}}{h _ {1}} \right| \int_ {\mathbb {R} ^ {p}} \mathcal {K} (v) | | v | | d v \\ \leq C _ {1} \frac {\left| h _ {1} - h _ {2} \right|}{h _ {1} ^ {p + 1}} \\ = O \left(\frac {\left| \kappa - \widehat {\kappa} \right|}{\kappa^ {\frac {p + 1}{r}}} \kappa^ {\frac {1}{r} - 1}\right) = O \left(\frac {\left| \kappa - \widehat {\kappa} \right|}{\kappa^ {\frac {p}{r} + 1}}\right). \\ \end{array}
$$
Similarly, we have,
$$
\begin{array}{l} I _ {2} = \int_ {\mathbb {R} ^ {p}} \left(\frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right)\right) ^ {2} - \frac {1}{h _ {1} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {1}}\right) \frac {1}{h _ {2} ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h _ {2}}\right) d x \\ = \frac {1}{h _ {2} ^ {p}} \int_ {\mathbb {R} ^ {p}} \mathcal {K} (v) \left(\mathcal {K} (v) - \mathcal {K} (v \frac {h _ {1}}{h _ {2}})\right) d v \leq C \frac {1}{h _ {2} ^ {p}} \left| 1 - \frac {h _ {1}}{h _ {2}} \right| \int_ {\mathbb {R} ^ {p}} \mathcal {K} (v) | | v | | d v = O \left(\frac {| \kappa - \widehat {\kappa} |}{\kappa^ {\frac {p}{r} + 1}}\right). \\ \end{array}
$$
I.1 Multivariate change point detection lemmas
We present some technical results corresponding to the generalization of the univariate CUSUM to the Multivariate case. For more details, we refer the interested readers to Padilla et al. (2021) and Wang et al. (2020).
Let  $\{X_{t}\}_{t = 1}^{T}\subset \mathbb{R}^{p}$  a process with unknown densities  $\{f_t\}_{t = 1}^T$
Assumption 5. We assume there exist  $\{\eta_k\}_{k=1}^K \subset \{2, \dots, T\}$  with  $1 = \eta_0 < \eta_1 < \dots < \eta_k \leq T < \eta_{K+1} = T + 1$ , such that
$$
f _ {t} \neq f _ {t + 1} \text {i f a n d o n l y i f} t \in \left\{\eta_ {1}, \dots , \eta_ {K} \right\}, \tag {55}
$$
Assume
$$
\begin{array}{l} \min  _ {k = 1, \dots , K + 1} \left(\eta_ {k} - \eta_ {k - 1}\right) \geq \Delta > 0, \\ 0 <   \left\| f _ {\eta_ {k + 1}} - f _ {\eta_ {k}} \right\| _ {L _ {\infty}} = \kappa_ {k} \text {f o r a l l} k = 1, \dots , K. \\ \end{array}
$$
In the rest of this section, we use the notation
$$
\widetilde {f} _ {t} ^ {(s, e ]} (x) = \sqrt {\frac {e - t}{(e - s) (t - s)}} \sum_ {j = s + 1} ^ {t} f _ {j} (x) - \sqrt {\frac {t - s}{(e - s) (e - t)}} \sum_ {j = t + 1} ^ {e} f _ {j} (x),
$$
for all  $0 \leq s < t < e \leq T$  and  $x \in \mathbb{R}^p$ .
Lemma 7. If  $[s, e]$  contain two and only two change points  $\eta_r$  and  $\eta_{r+1}$ , then
$$
\sup _ {s \leq t \leq e} | | \widetilde {f} _ {t} ^ {s, e} | | _ {L _ {2}} \leq \sqrt {e - \eta_ {r + 1}} | | f _ {r + 1} - f _ {r} | | _ {L _ {2}} + \sqrt {\eta_ {r} - s} | | f _ {r} - f _ {r - 1} | | _ {L _ {2}}.
$$
Proof. This is Lemma 15 in Wang et al. (2020). Consider the sequence  $\left\{g_t\right\}_{t = s + 1}^e$  be such that
$$
g _ {t} = \left\{ \begin{array}{l l} f _ {\eta_ {k}}, & \text {i f} s + 1 \leq t <   \eta_ {k}, \\ f _ {t}, & \text {i f} \eta_ {k} \leq t \leq e. \end{array} \right.
$$
For any  $t\geq \eta_k$
$$
\begin{array}{l} \widetilde {f} _ {t} ^ {s, e} - \widetilde {g} _ {t} ^ {s, e} \\ = \sqrt {\frac {e - t}{(e - s) (t - s)}} \left(\sum_ {i = s + 1} ^ {t} f _ {i} - \sum_ {i = s + 1} ^ {\eta_ {k}} f _ {\eta_ {k}} - \sum_ {i = \eta_ {k} + 1} ^ {t} f _ {i}\right) \\ - \sqrt {\frac {t - s}{(e - s) (e - t)}} \left(\sum_ {i = t + 1} ^ {e} f _ {i} - \sum_ {i = t + 1} ^ {e} f _ {i}\right) \\ = \sqrt {\frac {e - t}{(e - s) (t - s)}} \Big (\eta_ {k} - s \Big) \Big (f _ {\eta_ {k}} - f _ {\eta_ {k - 1}} \Big). \\ \end{array}
$$
So for  $t \geq \eta_k, ||\widetilde{f}_t^{s,e} - \widetilde{g}_t^{s,e}||_{L_2} \leq \sqrt{\eta_k - s}\kappa_k$ . Since  $\sup_{s \leq t \leq e} ||\widetilde{f}_t^{s,e}||_{L_2} = \max \left\{||\widetilde{f}_{\eta_k}^{s,e}||_{L_2}, ||\widetilde{f}_{\eta_{k+1}}^{s,e}||_{L_2}\right\}$ , and that
$$
\begin{array}{l} \max  \left\{\left| \left| \widetilde {f} _ {\eta_ {k}} ^ {s, e} \right| \right| _ {L _ {2}}, \left| \left| \widetilde {f} _ {\eta_ {k + 1}} ^ {s, e} \right| \right| _ {L _ {2}} \right\} \leq \sup  _ {s \leq t \leq e} \left| \left| \widetilde {g} _ {t} ^ {s, e} \right| \right| _ {L _ {2}} + \sqrt {\eta_ {k} - s} \kappa_ {k} \\ \leq \sqrt {e - \eta_ {k + 1}} \kappa_ {k + 1} + \sqrt {\eta_ {r} - s} \kappa_ {k} \\ \end{array}
$$
where the last inequality follows from the fact that  $g_{t}$  has only one change point in  $[s, e]$ .
Lemma 8. Suppose  $e - s \leq C_R\Delta$ , where  $C_R > 0$  is an absolute constant, and that
$$
\eta_ {k - 1} \leq s \leq \eta_ {k} \leq \dots \leq \eta_ {k + q} \leq e \leq \eta_ {k + q + 1}, \quad q \geq 0
$$
$$
\kappa_ {\max } ^ {s, e} = \max  \left\{\sup  _ {x \in \mathbb {R} ^ {p}} \left| f _ {\eta_ {p}} (x) - f _ {\eta_ {p - 1}} (x) \right|: k \leq p \leq k + q \right\}.
$$
Then for any  $k - 1 \leq p \leq k + q$ , it holds that
$$
\sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{e - s} \sum_ {i = s + 1} ^ {e} f _ {i} (x) - f _ {\eta_ {p}} (x) \right| \leq C _ {R} \kappa_ {\max } ^ {s, e}.
$$
Proof. This is Lemma 18 in Wang et al. (2020). Since  $e - s \leq C_R\Delta$ , the interval  $[s,e]$  contains at most  $C_R + 1$  change points. Observe that
$$
\begin{array}{l} \left| \left| \frac {1}{e - s} \sum_ {i = s} ^ {e} f _ {i} - f _ {\eta_ {p}} \right| \right| _ {L _ {\infty}} \\ = \frac {1}{e - s} \left| \left| \sum_ {i = s} ^ {\eta_ {k}} \left(f _ {\eta_ {k - 1}} - f _ {\eta_ {p}}\right) + \sum_ {i = \eta_ {k} + 1} ^ {\eta_ {k + 1}} \left(f _ {\eta_ {k}} - f _ {\eta_ {p}}\right) + \dots + \sum_ {i = \eta_ {k + q} + 1} ^ {e} \left(f _ {\eta_ {k + q}} - f _ {\eta_ {p}}\right) \right| \right| _ {L _ {\infty}} \\ \leq \frac {1}{e - s} \sum_ {i = s} ^ {\eta_ {k}} | p - k | \kappa_ {\max} ^ {s, e} + \sum_ {i = \eta_ {k} + 1} ^ {\eta_ {k + 1}} | p - k - 1 | \kappa_ {\max} ^ {s, e} + \ldots + \sum_ {i = \eta_ {k + q} + 1} ^ {e} | p - k - q - 1 | \kappa_ {\max} ^ {s, e} \\ \leq \frac {1}{e - s} \sum_ {i = s} ^ {e} \left(C _ {R} + 1\right) \kappa_ {\max } ^ {s, e}, \\ \end{array}
$$
where  $\left| p_1 - p_2 \right| \leq C_R + 1$  for any  $\eta_{p_1}, \eta_{p_2} \in [s, e]$  is used in the last inequality.
Lemma 9. Let  $(s,e)\subset (0,n)$  contains two or more change points such that
$$
\eta_ {k - 1} \leq s \leq \eta_ {k} \leq \dots \leq \eta_ {k + q} \leq e \leq \eta_ {k + q + 1}, \quad q \geq 1
$$
If  $\eta_{k} - s\leq c_{1}\Delta$  , for  $c_{1} > 0$  , then
$$
\left| \left| \widetilde {f} _ {\eta_ {k}} ^ {s, e} \right| \right| _ {L _ {\infty}} \leq \sqrt {c _ {1}} \left| \left| \widetilde {f} _ {\eta_ {k + 1}} ^ {s, e} \right| \right| _ {L _ {\infty}} + 2 \kappa_ {k} \sqrt {\eta_ {k} - s}
$$
Proof. This is Lemma 20 in Wang et al. (2020). Consider the sequence  $\left\{g_t\right\}_{t = s + 1}^e$  be such that
$$
g _ {t} = \left\{ \begin{array}{l l} f _ {\eta_ {r + 1}}, & s + 1 \leq t \leq \eta_ {k}, \\ f _ {t}, & \eta_ {k} + 1 \leq t \leq e \end{array} \right.
$$
For any  $t \geq \eta_r$ , it holds that
$$
| | \widetilde {f} _ {\eta_ {k}} ^ {s, e} - \widetilde {g} _ {\eta_ {k}} ^ {s, e} | | _ {L _ {\infty}} = \left| \left| \sqrt {\frac {(e - s) - t}{(e - s) (t - s)}} \Big (\eta_ {k} - s \Big) \Big (f _ {\eta_ {k + 1}} - f _ {\eta_ {k}} \Big) \right| \right| _ {L _ {\infty}} \leq \sqrt {\eta_ {k} - s} \kappa_ {k}.
$$
$$
\begin{array}{l} | | \widetilde {f} _ {\eta_ {k}} ^ {s, e} | | _ {L _ {\infty}} \leq | | \widetilde {g} _ {\eta_ {k}} ^ {s, e} | | _ {L _ {\infty}} + \sqrt {\eta_ {k} - s} \kappa_ {k} \leq \sqrt {\frac {\left(\eta_ {k} - s\right) \left(e - \eta_ {k + 1}\right)}{\left(\eta_ {k + 1} - s\right) \left(e - \eta_ {k}\right)}} | | \widetilde {g} _ {\eta_ {k + 1}} ^ {s, e} | | _ {L _ {\infty}} + \sqrt {\eta_ {k} - s} \kappa_ {k} \\ \leq \sqrt {\frac {c _ {1} \Delta}{\Delta}} | | \widetilde {g} _ {\eta_ {k + 1}} ^ {s, e} | | _ {L _ {\infty}} + \sqrt {\eta_ {k} - s} \kappa_ {k} \leq \sqrt {c _ {1}} | | \widetilde {f} _ {\eta_ {k + 1}} ^ {s, e} | | _ {L _ {\infty}} + 2 \sqrt {\eta_ {k} - s} \kappa_ {k}, \\ \end{array}
$$
where the first inequality follows from the observation that the first change point of  $g_{t}$  in  $(s, e)$  is at  $\eta_{k+1}$ .
Lemma 10. Under Assumption 5, for any interval  $(s,e)\subset (0,T)$  satisfying
$$
\eta_ {k - 1} \leq s \leq \eta_ {k} \leq \dots \leq \eta_ {k + q} \leq e \leq \eta_ {k + q + 1}, \quad q \geq 0.
$$
$$
b\in \operatorname *{arg  max}_{t = s + 1,\ldots ,e}\sup_{x\in \mathbb{R}^{p}}\Bigl|\widetilde{f}_{t}^{(s,e]}(x)\Bigr|.
$$
Then  $b \in \left\{\eta_1, \ldots, \eta_K\right\}$ . For any fixed  $z \in \mathbb{R}^p$ , if  $\widetilde{f}_t^{(s,e]}(z) > 0$  for some  $t \in (s,e)$ , then  $\widetilde{f}_t^{(s,e]}(z)$  is either strictly monotonic or decreases and then increases within each of the interval  $(s,\eta_k), (\eta_k,\eta_{k+1}), \ldots, (\eta_{k+q},e)$ .
Proof. We prove this by contradiction. Assume that  $b \notin \{\eta_1, \dots, \eta_K\}$ . Let  $z_1 \in \underset{x \in \mathbb{R}^p}{\arg \max} \left| \bar{f}_b^{s,e}(x) \right|$ . Due to the definition of  $b$ , we have
$$
b \in \operatorname * {a r g   m a x} _ {t = s + 1, \ldots , e} \left| \widetilde {f} _ {t} ^ {(s, e ]} \left(z _ {1}\right) \right|.
$$
It is easy to see that the collection of change points  $\{f_t(z_1)\}_{t=s+1}^e$  is a subset of the change points of  $\{f\}_{t=s+1}^e$ . Then, from Lemma 2.2 in Venkatraman (1992) that
$$
\widetilde {f} _ {b} ^ {(s, e ]} (z _ {1}) <   \max _ {j \in \{k, \dots , k + q \}} \widetilde {f} _ {\eta_ {j}} ^ {(s, e ]} (z _ {1}) \leq \max _ {t = s + 1, \dots , e} \sup _ {x \in \mathbb {R} ^ {p}} \left| \widetilde {f} _ {t} ^ {(s, e ]} (x) \right|
$$
which is a contradiction.
Recall that in Algorithm 1, when searching for change points in the interval  $(s,e)$ , we actually restrict to values  $t\in \left(s + \rho ,e - \rho\right)$ . We now show that for intervals satisfying condition  $SE$  from Lemma 1, taking the maximum of the CUSUM statistic over  $\left(s + \rho ,e - \rho\right)$  is equivalent to searching on  $(s,e)$ , when there are change points in  $\left(s + \rho ,e - \rho\right)$ .
Lemma 11. Let  $z_0 \in \mathbb{R}^p$ ,  $(s, e) \subset (0, T)$ . Suppose that there exists a true change point  $\eta_k \in (s, e)$  such that
$$
\min  \left\{\eta_ {k} - s, e - \eta_ {k} \right\} \geq c _ {1} \Delta , \tag {56}
$$
$$
\left| \widetilde {f} _ {\eta_ {k}} ^ {(s, e ]} \left(z _ {0}\right) \right| \geq \left(c _ {1} / 2\right) \frac {\kappa \Delta}{\sqrt {e - s}}, \tag {57}
$$
where  $c_{1} > 0$  is a sufficiently small constant. In addition, assume that
$$
\max  _ {t = s + 1, \dots , e} \left| \widetilde {f} _ {t} ^ {(s, e ]} \left(z _ {0}\right) \right| - \left| \widetilde {f} _ {\eta_ {k}} ^ {(s, e ]} \left(z _ {0}\right) \right| \leq c _ {2} \Delta^ {4} (e - s) ^ {- 7 / 2} \kappa , \tag {58}
$$
where  $c_2 > 0$  is a sufficiently small constant. Then for any  $d \in (s, e)$  satisfying
$$
\left| d - \eta_ {k} \right| \leq c _ {1} \Delta / 3 2, \tag {59}
$$
it holds that
$$
\left| \widetilde {f} _ {\eta_ {k}} ^ {(s, e ]} (z _ {0}) \right| - \left| \widetilde {f} _ {d} ^ {(s, e ]} (z _ {0}) \right| > c | d - \eta_ {k} | \Delta \left| \widetilde {f} _ {\eta_ {k}} ^ {(s, e ]} (z _ {0}) \right| (e - s) ^ {- 2}, \tag {60}
$$
where  $c > 0$  is a sufficiently small constant, depending on all the other absolute constants.