SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Carlos Misael Madrid Padilla
cmadridp@nd.edu
Haotian Xu
haotian.xu.1@warwick.ac.uk
Oscar Hernan Madrid Padilla
oscar.madrid@stat.ucla.edu
Daren Wang
dwang24@nd.edu
Yi Yu
yi.yu.2@warwick.ac.uk
This paper addresses the problem of localizing and inferring multiple change points, in non-parametric multivariate time series settings. Specifically, we consider a multivariate time series with potentially short-range dependence, whose underlying distributions have Hölder smooth densities and can change over time in a piecewise-constant manner. The change points, which correspond to the times when the distribution changes, are unknown. We present the limiting distributions of the change point estimators under the scenarios where the minimal jump size vanishes or remains constant. Such results have not been revealed in the literature in non-parametric change point settings. As byproducts, we develop a sharp estimator that can accurately localize the change points in multivariate non-parametric time series, and a consistent block-type long-run variance estimator. Numerical studies are provided to complement our theoretical findings.
Given a time series  $\{X_{t}\}_{t = 1}^{T}\subset \mathbb{R}^{p}$ , which is assumed to be an  $\alpha$ -mixing sequence of random vectors with unknown marginal distributions  $\{P_t\}_{t = 1}^T$ . To incorporate the nonstationarity of  $\{X_{t}\}_{t = 1}^{T}$ , we assume that there exists  $K\in \mathbb{N}$  change points, namely  $\{\eta_k\}_{k = 1}^K\subset \{2,\dots,T\}$  with  $1 = \eta_0 < \eta_1 < \ldots < \eta_k\leq T < \eta_{K + 1} = T + 1$ , such that
$$
P _ {t} \neq P _ {t - 1} \text {i f a n d o n l y i f} t \in \left\{\eta_ {1}, \dots , \eta_ {K} \right\}. \tag {1}
$$
Our primary interest is to develop accurate estimators of  $\{\eta_k\}_{k=1}^K$  and study their limiting properties. We refer to Assumption 1 for detailed technical conditions of our non-parametric change point model.
Nonstationary multivariate data are frequently encountered in real-world applications, including biology (e.g. Molenaar et al. 2009, Wolkovich & Donahue 2021), epidemiology (e.g. Azhar et al. 2021, Nguyen et al. 2021), social science (e.g. Kunitomo & Sato 2021, Cai et al. 2022), climatology (e.g. Corbella & Stretch 2012, Heo & Manuel 2022), finance (e.g. Herzel et al. 2002, Schmitt et al. 2013), neuroscience (e.g. Gorrostieta et al. 2019, Frolov et al. 2020), among others.
Due to the importance of modeling nonstationary data in various scientific fields, we have witnessed a soaring growth of statistical change point literature, (e.g. Aue et al. 2009, Fryzlewicz 2014, Cho & Fryzlewicz 2015, Cho 2016, Wang et al. 2020, Padilla et al. 2022). However, there are a few limitations in the existing works on multivariate non-parametric settings. Firstly, to the best of our knowledge, temporal dependence, which commonly appears in time series, has not been considered. Secondly, there is no localization consistency result for data with the underlying densities being Hölder smooth with arbitrary degree of smoothness. Lastly and most importantly, the limiting distributions of change point estimators and the asymptotic inference for change points have not been well studied.
Taking into account the aforestated limitations, this paper examines change point problems in a fully non-parametric time series framework, wherein the underlying distributions are only assumed to have Hölder smooth continuous densities and can change over time in a piecewise constant manner. The rest of the paper is organized as follows. In Section 2, we explain the model assumptions for multivariate time series with change points in a non-parametric setting. Section 3 details the two-step change point estimation procedure, as well as the estimators at each step. Theoretical results, including the consistency of the preliminary estimator and the limiting distribution of the final estimator, are presented in Section 4. Section 5 evaluates the practical performance of the proposed procedure via various simulations and a real data analysis. Finally, Section 6 concludes with a discussion.
Notation. For any function \( f: \mathbb{R}^p \to \mathbb{R} \) and for \( 1 \leq q < \infty \), define \( \|f\|_{L_q} = (\int_{\mathbb{R}^p} |f(x)|^q dx)^{1/q} \) and for \( q = \infty \), define \( \|f\|_{L_\infty} = \sup_{x \in \mathbb{R}^p} |f(x)| \). Define \( L_q = \{f: \mathbb{R}^p \to \mathbb{R}, \|f\|_q < \infty\} \). Moreover, for \( q = 2 \), define \( \langle f, g \rangle_{L_2} = \int_{\mathbb{R}^p} f(x)g(x)dx \) where \( f, g: \mathbb{R}^p \to \mathbb{R} \). For any vector \( s = (s_1, \ldots, s_p)^\top \in \mathbb{N}^p \), define \( |s| = \sum_{i=1}^{p} s_i \), \( s! = s_1! \cdots s_p! \) and the associated partial differential operator \( D^s = \frac{\partial^{|s|}}{\partial x_1^{s_1} \cdots \partial x_p^{s_p}} \). For \( \alpha > 0 \), denote \( \lfloor \alpha \rfloor \) to be the largest integer smaller than \( \alpha \). For any function \( f: \mathbb{R}^p \to \mathbb{R} \) that is \( \lfloor \alpha \rfloor \)-times continuously differentiable at point \( x_0 \), denote by \( f_{x_0}^\alpha \) its Taylor polynomial of degree \( \lfloor \alpha \rfloor \) at \( x_0 \), which is defined as \( f_{x_0}^\alpha(x) = \sum_{|s| \leq \lfloor \alpha \rfloor} (x - x_0)^s / s! D^s f(x_0) \). For a constant \( L > 0 \), let \( \mathcal{H}^\alpha(L, \mathbb{R}^p) \) be the set of functions \( f: \mathbb{R}^p \to \mathbb{R} \) such that \( f \) is \( \lfloor \alpha \rfloor \)-times differentiable for all \( x \in \mathbb{R}^p \) and satisfy \( |f(x) - f_{x_0}^\alpha(x)| \leq L|x - x_0|^\alpha \), for all \( x, x_0 \in \mathbb{R}^p \). Here \( |x - x_0| \) is the Euclidean distance between \( x, x_0 \in \mathbb{R}^p \). In non-parametric statistics literature, \( H^\alpha(L, \mathbb{R}^p) \) is often referred to as the class of Hölder functions. For two positive sequences \( \{a_n\}_{n \in \mathbb{N}^+} \) and \( \{b_n\}_{n \in \mathbb{N}^+} \), we write \( a_n = O(b_n) \) or \( a_n \lesssim b_n \), if \( a_n \leq C b_n \) with some constant \( C > 0 \) that does not depend on \( n \), and \( a_n = O(b_n) \) or \( a_n > b_n \), if \( a_n = O(b_n) \) and \( b_n = O(a_n) \). For a deterministic or random \( R \)-valued sequence \( a_n \), write that a sequence of random variable \( X_n = O_p(a_n) \), if \( \lim_{M \to \infty} \lim_{n \to \infty} P(|X_n| > M a_n) = 0 \). Write \( X_n = o_p(a_n) \) if \( \lim_{n \to \infty} P(|X_n| > M a_n) = 0 \) for all \( M > 0 \). The convergences in distribution and probability are respectively denoted by \( P_i' \) and \( P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' - P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' + P_i' = 0.
2 Model setup
Detailed assumptions imposed on the model (1) are collected in Assumption 1.
Assumption 1. The data  $\{X_{t}\}_{t = 1}^{T}\subset \mathbb{R}^{p}$  are generated based on model (1) and satisfy the following. a. For each  $t = \{1,\dots ,T\}$ , the distribution  $P_{t}$  has a Lebesgue density function  $f_{t}:\mathbb{R}^{p}\to \mathbb{R}$ , such that  $f_{t}\in \mathcal{H}^{r}(L,\mathcal{X})$  with  $r,L > 0$ , where  $\mathcal{X}$  is the union of the supports of all  $f_{t}$ , and  $\mathcal{X}$  has bounded Lebesgue measure.
b. Let  $g_{t}$  be the joint density of  $X_{1}$  and  $X_{t + 1}$  . It satisfies that  $\| g_t\|_{L_\infty} <   \infty$    
c. The minimal spacing between two consecutive change points  $\Delta = \min_{k = 1}^{K + 1}(\eta_k - \eta_{k - 1}) > 0$    
d. The minimal jump size between two consecutive change points  $\kappa = \min_{k = 1,\dots ,K}\kappa_{k} > 0$  where  $\kappa_{k} = \| f_{\eta_{k}} - f_{\eta_{k + 1}}\|_{L2}$  denotes the jump size at the kth change point.   
e. The process  $\{X_{t}\}_{t\in \mathbb{Z}}$  is  $\alpha$  -mixing with mixing coefficients
$$
\alpha_ {k} = \sup  _ {t \in \mathbb {Z}} \alpha (\sigma (X _ {s}, s \leq t), \sigma (X _ {s}, s \geq t + k)) \leq e ^ {- 2 c k} \quad f o r a l l k \in \mathbb {Z}. \tag {2}
$$
The minimal spacing  $\Delta$  and the minimal jump size  $\kappa$  are two key parameters characterizing the change point phenomenon. Assumption 1d. characterizes the changes in density functions through the function's  $L_{2}$ -norm, enabling us to detect local and global changes in non-parametric settings.
The decay rate of  $\alpha_{k}$  in Assumption 1e. imposes an upper bound on the temporal dependence. This is a standard requirement in the literature (e.g Abadi 2004, Merleved et al. 2009).
Revolving the change point estimators, we are to conduct the estimation and inference tasks. For a sequence of estimators  $\widehat{\eta}_1 < \ldots < \widehat{\eta}_{\widehat{K}} \subset \{1, \ldots, T\}$ , our first task is to show the localization consistency, i.e. with probability tending to one as the sample size  $T$  grows unbounded, it holds that
$$
\widehat {K} = K \text {a n d} \max  _ {k = 1, \dots , \widehat {K}} | \widehat {\eta} _ {k} - \eta_ {k} | \leq \epsilon , \text {w i t h} \lim  _ {T \rightarrow \infty} \frac {\epsilon}{\Delta} = 0. \tag {3}
$$
We refer to  $\epsilon$  as the localization error in the rest of this paper.
With a consistent estimation result, we further refine  $\{\widehat{\eta}_k\}_{k=1}^{\widehat{K}}$  and obtain  $\{\widetilde{\eta}_k\}_{k=1}^{\widehat{K}}$  with error bounds  $|\widetilde{\eta}_k - \eta_k| = O_p(1)$  and derive the limiting distribution of  $(\widetilde{\eta}_k - \eta_k)\kappa_k^{\frac{p}{r} + 2}$ .
We briefly summarize the contributions of our paper as follows.
- We develop a multivariate non-parametric seeded change point detection algorithm detailed Algorithm 1, which is based on the seeded binary segmentation method (SBS), proposed in Kovacs et al. (2020) in the univariate setting. To the best of our knowledge, we are the first to innovatively adapt SBS to a multivariate non-parametric change point model.
- Under the signal-to-noise ratio condition in Assumption 3 that  $\kappa^2\Delta \gtrsim \log (T)T^{p / (2r + p)}$  we demonstrate that the output of Algorithm 1 is consistent, with localization errors  $\epsilon \asymp \kappa_k^{-2}T^{p / (2r + p)}\log (T)$ , for  $k\in \{1,\dots ,K\}$ . This localization error is first obtained for  $\alpha$ -mixing time series with a generic smoothness assumption, while the state-of-the-art method from Padilla et al. (2021) only focuses on Lipschitz smooth densities and under temporal independence.
- Based on the consistent estimators  $\{\widehat{\eta}\}_{k=1}^{K}$ , we construct the refined estimators  $\{\widetilde{\eta}_k\}_{k=1}^{K}$  and derive their limiting distributions in different regimes, as detailed in Theorem 2. These results are novel in the literature of change point and time series analysis.
- Extensive numerical results are presented in Section 5 to corroborate the theoretical findings. The code used for numerical experiments is available upon request prior to publication.
3 A two-step multivariate non-parametric change point estimators
In this section, we present the initial and refined change point estimators, both of which share the same building block, namely the non-parametric CUSUM statistic.
Definition 1 (Non-parametric CUSUM statistic). For any integer triplet  $0 \leq s < t < e \leq T$ , let the CUSUM statistic be
$$
\widetilde {F} _ {t, h} ^ {(s, e ]} (x) = \sqrt {\frac {e - t}{(e - s) (t - s)}} \sum_ {i = s + 1} ^ {t} F _ {i, h} (x) - \sqrt {\frac {t - s}{(e - s) (e - t)}} \sum_ {i = t + 1} ^ {e} F _ {i, h} (x), x \in \mathbb {R} ^ {p},
$$
where  $F_{t,h}$  is a kernel estimator of  $f_{t}$ , i.e.  $F_{t,h}(x) = \mathcal{K}_h(x - X_t)$  with the kernel function
$$
\mathcal {K} _ {h} (x) = \frac {1}{h ^ {p}} \mathcal {K} \left(\frac {x}{h}\right), \quad x \in \mathbb {R} ^ {p},
$$
accompanied with the bandwidth  $h > 0$ .
The CUSUM statistic is a key ingredient of our algorithm and is based on the kernel estimator  $F_{t,h}(\cdot)$ . We highlight that kernel-based change-point estimation techniques have been employed in detecting change points in non-parametric models in existing literature, as demonstrated in, for instance, Arlot et al. (2019), Li et al. (2019), Padilla et al. (2021).
Our preliminary estimator is obtained by combining the CUSUM statistic in Definition 1 with a modified version of SBS based on a collection of deterministic seeded intervals defined in Definition 2.
Definition 2 (Seeded intervals). Let  $\Re = \left\lceil C_{\Re}\log_2\left(\frac{T}{\Delta}\right)\right\rceil$ , with some sufficiently large absolute constant  $C_{\Re} > 0$ . For  $k\in \{1,\ldots ,\Re \}$ , let  $\mathcal{I}_k$  be the collection of  $2^{k} - 1$  intervals of length  $l_{k} = T2^{-k + 1}$  that are evenly shifted by  $l_{k} / 2 = T2^{-k}$ , i.e.
$$
\mathcal {J} _ {k} = \left\{ \right.\left( \right.\left\lfloor (i - 1) T 2 ^ {- k} \right\rfloor , \left\lceil (i - 1) T 2 ^ {- k} + T 2 ^ {- k + 1} \right\rceil \left. \right], \quad i = 1, \dots , 2 ^ {k} - 1 \left. \right\}.
$$
The overall collection of seeded intervals is denoted as  $\mathcal{J} = \cup_{k=1}^{\Re} \mathcal{J}_k$ .
With the CUSUM statistics and the seeded intervals as building blocks, we are now ready to present our multivariate non-parametric seeded change point detection algorithm.
Algorithm 1 Multivariate non-parametric Seeded Binary Segmentation. MNSBS  $((s,e),\mathcal{J},\tau ,h)$
INPUT: Sample  $\{X_{t}\}_{t = s}^{e}\subset \mathbb{R}^{p}$  collection of seeded intervals  $\mathcal{I}$  tuning parameter  $\tau >0$  and bandwidth  $h > 0$    
initialization: If  $(s,e] = (0,n]$  , set  $\mathbf{S}\rightarrow \emptyset$  and set  $\rho \to \log (T)h^{-p}$    
for  $\mathcal{I} = (\alpha ,\beta ]\in \mathcal{J}$  do if  $\mathcal{I} = (\alpha ,\beta ]\subseteq (s,e]$  and  $\beta -\alpha >2\rho$  then  $b_{\mathcal{I}}\gets \arg \max_{\alpha +\rho \leq t\leq \beta -\rho}\| \widetilde{F}_{t,h}^{(\alpha ,\beta)}\|_{L_2}$ $a_{\mathcal{I}}\gets \| \widetilde{F}_{b_{\mathcal{I}},h}^{(\alpha ,\beta)}\|_{L_2}$  else  $a_{\mathcal{I}}\gets -1$  end if   
end for   
 $\mathcal{M}^{s,e} = \{\mathcal{I}:a_{\mathcal{I}} > \tau \}$    
if  $\mathcal{M}^{s,e}\neq \emptyset$  then  $\mathcal{I}^* \leftarrow \arg \min_{\mathcal{I}\in \mathcal{M}^{s,e}}|\mathcal{I}|$  S  $\leftarrow$  S U  $\{b_{\mathcal{I}^*}\}$  MNSBS((s,bI\*),J,τ,h) MNSBS((bI\*+1,e),J,τ,h)   
end if
OUTPUT: The set of estimated change points  $\mathbf{S}$
Algorithm 1 presents a methodological approach to addressing the problem of estimating multiple change points in multivariate time series data. At its core, the algorithm leverages the strength of seeded intervals, forming a multi-scale search mechanism. To identify potential change points, the method recursively employs the CUSUM statistics. For the functionality of the algorithm, specific inputs are required. These include the observed data set, represented as  $X_{t t = 1}^{T}$ , the seeded intervals denoted by  $\mathcal{I}$ , the bandwidth  $h$  that is crucial for constructing the CUSUM statistics, and a threshold,  $\tau$ , which is instrumental in change point detection. We provide theoretical and numerical guidance for tuning parameters in Sections 4 and 5, respectively.
Delving deeper into the architecture of Algorithm 1, it becomes evident that the SBS functions as its foundational framework, while the nonparametric version of the CUSUM statistics acts as its functional units. The design of this algorithm is particularly tailored given its inclination toward nonparametric detection and its ability to identify multiple change points. The SBS is, in essence, an advanced version of the moving-window scanning technique. Its distinctive characteristic is its adaptability in handling the challenges posed by multiple change points that exhibit unpredictable spacing. Instead of being confined to a fixed window width, the SBS introduces versatility by incorporating a range of window width options. Each of these widths is methodically applied during a moving-window scan.
Based on the preliminary estimators  $\{\widehat{\eta}_k\}_{k=1}^{\widehat{K}}$  provided by Algorithm 1, we further develop a refinement procedure to enhance the localization accuracy. To be more specific, let
$$
s _ {k} = \frac {9}{1 0} \widehat {\eta} _ {k - 1} + \frac {1}{1 0} \widehat {\eta} _ {k} \quad \text {a n d} \quad e _ {k} = \frac {9}{1 0} \widehat {\eta} _ {k + 1} + \frac {1}{1 0} \widehat {\eta} _ {k}. \tag {4}
$$
Then, the preliminary estimators  $\{\widehat{\eta}_k\}_{k=1}^{\widehat{K}}$  and  $\widetilde{h} \asymp h$  produce an estimator of  $\kappa_k$  as:
$$
\begin{array}{l} \widehat {\kappa} _ {k} = \left\| \sqrt {\frac {\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k}}{(\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k - 1}) (\widehat {\eta} _ {k} - \widehat {\eta} _ {k - 1})}} \sum_ {i = \widehat {\eta} _ {k - 1} + 1} ^ {\widehat {\eta} _ {k}} F _ {i, \widetilde {h}} - \sqrt {\frac {(\widehat {\eta} _ {k} - \widehat {\eta} _ {k - 1})}{(\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k - 1}) (\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k})}} \sum_ {i = \widehat {\eta} _ {k} + 1} ^ {\widehat {\eta} _ {k + 1}} F _ {i, \widetilde {h}} \right\| _ {L _ {2}} \\ \times \sqrt {\frac {\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k - 1}}{(\widehat {\eta} _ {k} - \widehat {\eta} _ {k - 1}) (\widehat {\eta} _ {k + 1} - \widehat {\eta} _ {k})}}. \tag {5} \\ \end{array}
$$
We then propose the final change points estimators as
$$
\widetilde {\eta} _ {k} = \underset {s _ {k} <   \eta <   e _ {k}} {\arg \min } \widehat {Q} _ {k} (\eta), \tag {6}
$$
$$
\widehat {Q} _ {k} (\eta) = \Big \{\sum_ {t = s _ {k} + 1} ^ {\eta} \| F _ {t, h _ {1}} - F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \| _ {L _ {2}} ^ {2} + \sum_ {t = \eta + 1} ^ {e _ {k}} \| F _ {t, h _ {1}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \| _ {L _ {2}} ^ {2} \Big \},
$$
with  $h_1 = c_{\widehat{\kappa}_k} \widehat{\kappa}_k^{1/r}$  and  $F_{(s,e],h_1} = \frac{1}{e - s} \sum_{i=s+1}^e F_{i,h_1}$  for integers  $e > s$ .
If the initial change point estimators are consistent, i.e. (3) holds with probability tending to 1, then the interval  $(\hat{\eta}_{k-1}, \hat{\eta}_{k+1})$  is anticipated to contain merely one undetected change point. By conservatively trimming this interval to  $(s_k, e_k)$ , we can safely any change points previously detected within  $(\hat{\eta}_{k-1}, \hat{\eta}_{k+1})$ . Consequently, the trimmed interval  $(s_k, e_k)$  contains only true change point  $\eta_k$  with high probability. Due to the same reason, our choice of weight in Equation 4,1/10, is a convenient choice. In general, any constant weight between 0 and  $1/2$  would suffice. Inspired by Padilla et al. (2021), who proposed to use  $O(\kappa_k)$  as an optimal bandwidth in the context of Lipschitz densities, we adopt  $h_1 = O\left(\hat{\kappa}_k^{1/r}\right)$  as the bandwidth for our kernel density estimator. This choice incorporates the broader scope of our work, which studies a more general degree of smoothness. Notably, if the underlying density functions strictly adhere to the Lipschitz criterion and  $r = 1$ , our bandwidth selection aligns with that recommended by Padilla et al. (2021). We would like to emphasize that while the procedure proposed by Padilla et al. (2021) required knowledge of the population quantities  $\kappa_k$ , our approach is adaptive as we provide data-driven methods to estimate  $\kappa_k$  accurately.
With our newly proposed estimators, in Theorem 2, we derive an improved error bound for the refined estimators  $\{\widetilde{\eta}_k\}_{k=1}^K$  over the preliminary estimators  $\{\widehat{\eta}_k\}_{k=1}^K$ . We also study the limiting distributions of the refined estimators. Section 4.4 and Section 5 will discuss the theoretically justified rates and practical choices of tuning parameters, respectively.
The computational complexity of Algorithm 1 is  $O(T^2 \log (T) \Delta^{-1} \cdot \text{Kernel})$ , where  $O(T^2 \log (T) \Delta^{-1})$  is due to the computational cost of the SBS, and "Kernel" stands for the computational cost of numerical computation of the  $L_2$ -norm of the CUSUM statistics based on the kernel function evaluated at each time point. The dependence on the dimension  $p$  is only through the evaluation of the kernel function. The computational complexity of the final estimators (including estimating  $\widehat{\kappa}_k$ 's) is  $O(T \cdot \text{Kernel})$ . Therefore, the overall cost for deriving  $\{\widetilde{\eta}_k\}_{k=1}^K$  is  $O(T^2 \log T \Delta^{-1} \cdot \text{Kernel})$ .
4 Consistent estimation and limiting distributions
To establish the theoretical guarantees of our estimators, we first state conditions needed for the kernel function  $\mathcal{K}(\cdot)$ .
Assumption 2 (The kernel function). Assume that the kernel function  $\mathcal{K}(\cdot)$  has compact support and satisfies the following additional conditions.
a. For the Hölder smooth parameter  $r$  in Assumption 1a, assume that  $\mathcal{K}(\cdot)$  is adaptive to  $\mathcal{H}^r (L,\mathbb{R}^p)$ , i.e. for any  $f\in \mathcal{H}^r (L,\mathbb{R}^p)$ , it holds that
$$
\sup  _ {x \in \mathbb {R} ^ {p}} \left| \int_ {\mathbb {R} ^ {p}} h ^ {- p} \mathcal {K} \left(\frac {x - z}{h}\right) f (z) d z - f (x) \right| \leq \widetilde {C} h ^ {r},
$$
for some absolute constant  $\widetilde{C} > 0$  and tuning parameter  $h > 0$ .
b. The class of functions  $\mathcal{F}_{\mathcal{K}} = \{\mathcal{K}(x - \cdot) / h : \mathbb{R}^p \to \mathbb{R}^+, h > 0\}$  is separable in  $L_{\infty}(\mathbb{R}^{p})$  and is a uniformly bounded VC-class; i.e. there exist constants  $A, \nu > 0$  such that for any probability measure  $Q$  on  $\mathbb{R}^p$  and any  $u \in (0, \| \mathcal{K}\|_{L_{\infty}})$ , it holds that  $\mathcal{N}(\mathcal{F}_{\mathcal{K}}, L_2(Q), u) \leq (A \| \mathcal{K}\|_{L_{\infty}} / u)^{\nu}$ , where  $\mathcal{N}(\mathcal{F}_{\mathcal{K}}, L_2(Q), u)$  denotes the  $u$ -covering number of the metric space  $(\mathcal{F}_{\mathcal{K}}, L_2(Q))$ .
c. Let  $m = \lfloor r\rfloor$  and it holds that  $\begin{array}{r}\int_0^\infty t^{m - 1}\sup_{\| x\| \geq t}|\mathcal{K}(x)|^m\mathrm{d}t <   \infty ,\int_{\mathbb{R}^p}\mathcal{K}(z)\| z\| \mathrm{d}z\leq C_K, \end{array}$  where  $C_K > 0$  is an absolute constant.
Assumption 2 is a standard condition in the non-parametric literature (e.g. Giné & Guillou 1999, 2001, Sriperumbudur & Steinwart 2012, Kim et al. 2019, Padilla et al. 2021) and holds for various kernels, such as the Triweight, Epanechnikov and Gaussian kernels, which are considered in Section 5.
4.1 Consistency of preliminary estimators
To establish the consistency of the preliminary estimators outputted by Algorithm 1, we impose the following signal-to-noise ratio condition.
Assumption 3 (Signal-to-noise ratio). Assume there exists an arbitrarily slow diverging sequence  $\gamma_T > 0$  such that
$$
\kappa^ {2} \Delta > \gamma_ {T} \log (T) T ^ {\frac {p}{2 r + p}}.
$$
We note that Assumption 3 is a mild condition, as it allows both the jump size  $\kappa$  to vanish asymptotically and/or the spacing  $\Delta$  between change points to be much smaller than  $T$ . The consistency of Algorithm 1 is established in the following theorem.
Theorem 1. Suppose Assumptions 1, 2 and 3 hold. Let  $\{\widehat{\eta}_k\}_{k=1}^K$  be the estimated change points returned by Algorithm 1 with tuning parameters  $\tau = c_{\tau} T^{p/(4r+2p)} \log^{1/2}(T)$  and  $h = c_h T^{-1/(2r+p)}$  for sufficiently large constants  $c_h, c_{\tau} > 0$ . Then
$$
\mathbb {P} \Big \{\widehat {K} = K, | \widehat {\eta_ {k}} - \eta_ {k} | \leq C _ {\epsilon} \kappa_ {k} ^ {- 2} T ^ {\frac {p}{2 r + p}} \log (T), \forall k = 1, \ldots , K \Big \} \geq 1 - 3 C _ {p, \mathcal {K}} T ^ {- 1},
$$
where  $C_{\epsilon}$  and  $C_{p,\mathcal{K}}$  are positive constants only depending on the kernel and the dimension  $p$ .
4.2 Refined estimators and their limiting distributions