Change point detection and inference in multivariate non-parametric models under mixing conditions
Authors: Carlos Misael Madrid Padilla
Haotian Xu
Daren Wang
OSCAR HERNAN MADRID PADILLA
Yi Yu
Venue: NeurIPS 2023 poster (2023)
Keywords: Multivariate; Nonparametric; Change point inference; short range dependence; Long-run variance; Confidence interval.

Abstract: This paper addresses the problem of localizing and inferring multiple change points, in non-parametric multivariate time series settings. Specifically, we consider a multivariate time series with potentially short-range dependence, whose underlying distributions have Hölder smooth densities and can change over time in a piecewise-constant manner. The change points, which correspond to the times when the distribution changes, are unknown.
 We present the limiting distributions of the change point estimators under the scenarios where the minimal jump size vanishes or remains constant.  Such results have not been revealed in the literature in non-parametric change point settings. As byproducts, we develop a sharp estimator that can accurately localize the change points in multivariate non-parametric time series, and a consistent block-type long-run variance estimator.  Numerical studies are provided to complement our theoretical findings.

Change point detection and inference in multivariate non-parametric models under mixing conditions
Carlos Misael Madrid Padilla Department of Mathematics University of Notre Dame cmadridpēnd edu
Haotian Xu
Daren Wang Department of Statistics University of Notre Dame dwang240nd. edu
Department Statistics University of Warwick
haotian xu wwwwwwwicc......m... ac. uk
Oscar Hernan Madrid Padilla
Yi Yu Department of Statistics University of Warwick yi yu 2ew.222wwwwiccc ac. uk
Department of Statistics
University of California, Los Angeles oscar madridestat. ucla. edu
Abstract
This paper addresses the problem of localizing and inferring multiple change points, in non-parametric multivariate time series settings. Specifically, consider a multivariate time series with potentially short-range dependence, whose underlying distributions have Hölder smooth densities and can change over time in piecewise- constant manner. The change points, which correspond to the times when the distribution changes, are unknown. We present the limiting distributions of the change point estimators under the scenarios where the minimal jump size vanishes or remains constant. Such results have not been revealed the literature in non- parametric change point settings. As byproducts, we develop a sharp estimator that can accurately localize the change points in multivariate non-parametric time series, and a consistent block-type long-run variance estimator. Numerical studies are provided to complement our theoretical findings.
Introduction
Given time series (X,)__ C TR,, which is assumed to be an a-mixing sequence of random vectors
with unknown marginal distributions (P:) To incorporate the nonstationarity of (X,)T= we assume that there exists K E N change points, namely {T J C (2, ...,. T') with 710 m < Th < T -K- T 1, such that
P. + Pt-1 and only if E (m,..., 773.
(1)
Our primary interest is develop accurate estimators of (nx k and study their limiting properties.
We refer to Assumption for detailed technical conditions of our non- non-parametric change point model.
Nonstationary multivariate data are frequently encountered in real-world applications, including biology (e.g. Molenaar et al. 2009 Wolkovich Donahue 2021), epidemiology (e.g. AAzhar et al.)
2021 Nguyen al. 2021) 1), social science (e.g. Kunitomo & Sato 2021] Cai et al. (022). climatology
(e.g. Corbella & Stretch 2012 Heo & Manuel 2022), finance (c.g. Herzel et al. 2002 Schmitt et al.
2013
neuroscience (c.g. Gorrosticta et al. 2019 Frolov al. 2020), among others.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Due to the importance of modeling nonstationary data in various scientific fields, we have witnessed a soaring growth of statistical change point literature, (c.g. Aue et al. 2009 Fryzlewicz 2014 Cho
& Fryzlewicz 2015 Cho 2016 Wang et al. 2020 Padilla et al. 2022) However, there are a few limitations the existing works on multivariate non-parametric settings. Firstly, to the best of our knowledge, temporal dependence, which commonly appears in time series, has not been considered. Secondly, there is no localization consistency result for data with the underlying densities being Hölder smooth with arbitrary degree of smoothness. Lastly and most importantly, the limiting distributions of change point estimators and the asymptotic inference for change points have not been well studied.
Taking into account the aforestated limitations, this paper examines change point problems in a fully non con-parametric time series framework, wherein the underlying distributions are only assumed to have Hölder smooth continuous densities and can change over time in a piecewise constant manner.
The rest of the paper is organized as follows. In Section 2 we explain the model assumptions for multivariate time series with change points well as the estimators at each step. Theoretical
non-parametric setting. Section [3 details the
two-step change point estimation procedure, results, including the consistency of the preliminary estimator and the limiting distribution of the final estimator, are presented in Section 4 Section [ evaluates the practical performance of the proposed procedure via various simulations and a real data analysis. Finally, Section 6 concludes with discussion.
Notation. For any function f: - R and for 1 S q- co, define [l,lle, (J&r (f(z) and for q oc, define || suPrER= [f()). Define Lg {ff 12 + E. ll oo). Moreover, for 9 2, define (f. L2 = Jzz f(z (z)g(()) ((x dx where f.g I² + R. For any vector Σm= S4, s! sn s.!...sp spl and the associated partial differential g1-1 FF For > 0, denote [a] to be the largest integer smaller than a. For any function / R - R that is |[x [a--ttimes continuously differentiable at point zo. denote by 5% its Taylor polynomial of degree |a] at zo, which is defined as f2: (2) = E (x - xo)) /s!""f(zo)
($1..... .p) E NP, define
operator D
0u Ou
Rp - R such that is |a| -times differentiable for all æ E IR and satisfy [f(x) - fag (x)1 S L| Tul "ol"" for all x, To E RR. Here Jx: xoo is the Fuclidean distance between xr 3o E TRP In non-parane non-parametric statistics literature, (L, is often referred to as the class of Hölder functions. For two positive sequences {an hrEN+ and {b, JoeNt , we write a, = (bb)) or an S bn if a, S Cb,, with some constant C' > that does not depend on n. and an = (bb,) or an bn if an = O(b,) and b, O(a,). For a deterministic or random R- Raluleed sequence an. write that a sequence of random variable X,, = (a,).. lim lim sup, P([ 2 Ma,) = O. Write X,, = op(an) if lim sup, P(|Xn 2 Ma,) = 0 for all M > 0. The convergences in distribution and probability are respectively denoted by 2, and 4.
For constant L 0, let 71 (L, R?) be the set of functions
2 Model setup
Detailed assumptions imposed on the model [ are collected in Assumption
Assumption 1. The data { [T_ C IR" are generated based on model ① and satisfy the following. For each (I,..., T", the distribution P. has Lebesgue density function f. IRP + IR, such that ft E H",L with r, L > 0, where X is the union of the supports of all So and X has bounded Lebesgue measure.
b. Let gt be the joint density of X1 and X,-1. It satisfies that [|ge I1n 00.
K+1 (M% mk 1) > O. c. The minimal spacing between two consecutive change points A = min 1i
d. The minimal jump size between two consecutive change points k mink=1.....K Kk > 0, where
KA = 1l Smee [lz, denotes the jump size at the kth change point.
The process (Xthez hez is a-mixing with mixing coefficients
0 = sup (X, ((), 2 1++R k)) S 1E2
for all ls E Z.
The minimal spacing A and the minimal jump size are two key parameters characterizing the change point phenomenon. Assumption Td. characterizes the changes in density functions through the function's 's norm, enabling us to detect local and global changes in non-parametric settings.

---

The decay rate of ay in Assumption Te. imposes an upper bound on the temporal dependence. This a standard requirement in the literature (e.g [Abadi 2004 Merlevède al. 20099)
Revolving the change point estimaaoors, we are to conduct the estimation and inference tasks. For sequence of estimators fi <... " (I,...: 7), our first task is to show the localization consistency, i.c. with probability tending to one as the sample size ' grows unbounded, it holds that
K = K and max
7hs
The S e, with lim T- A
I=....,
We refer to as the localization error in the rest of this paper.
With consistent estimation result, further refine {fk k== and obtain {7s k==1 with error bounds 7i
2+2
7 (1) and derive the limiting distribution of (7) )
We briefly summarize the contributions of paper as follows.
We develop a multivariate non- parametric seeded change point detection algorithm detailed Algo- rithm which is based on the seeded binary segmentation method (SBS), proposed in Kovács et al.
(2020 in the univariate setting. To the best of our knowledge, we are the first to innovatively adapt SBS to multivariate non-parametric change point model.
3 that AAA 2 log(T)TT//er+))) we demonstrate that the output of Algorithm [ is consistent, with localization errors e Kg x=22p/((r+p) log( for k E (L..... K. This localization error is first obtained for a-mixing time series with a generic smoothness assumption, while the state-of-the-ane method from Padilla et al. (202T) only focuses on Lipschitz smooth densities and under temporal independence.
Under the signal-to-noise ratio condition in Assumption
Based on the consistent estimators { R= {m)=_ we construct the refined estimators [7 } k= and derive
their limiting distributions in different regimes, detailed in Theorem 2
These results are novel in
the literature of change point and time series analysis.
Extensive numerical results are presented in Section corroborate the theoretical findings. The code used for numerical experiments is available upon request prior to publication.
3 A two-step multivariate non-parametric change point estimators
In this section, present the initial and refined change point estimators, both of which share the same building block, namely the non-parametric CUSUM statistic.
Definition -nrn-paraeetric CUSUM statistic). For any integer triplet S Le e T, let the CUSUM statistic be
F(a.e) (x) 1.h ==)=
e
Σ F.,(())
t-s
Σ Fa,()), EER.
(e- 5) i=s+1
V (e s)(c-tt) ist+1
where Fe, is a kernel estimator of fú. i.e. FL,(()) = Kn(x X1) with the kernel function
1
(x(x) = hv
1C
EER.
h
accompanied with the bandwidth h 0.
The CUSUM statistic is a key ingredient of our algorithm and is based on the kernel estimator Fr..(-). We highlight that kernel-based change-point estimation techniques have been employed detecting change points in non- parrameetiii models in existing literature, as demonstrated in, for instance, Arlot
et al. (2099) Li al. (2019), Padilla al. (202T).
Our preliminary estimator is obtained by combining the CUSUM statistic in Definition I with modified version of SBS based on collection of deterministic seeded intervals defined in Dehnition
[ with some sufficiently large absolute constant CR > O. For k. (1...., Rt), let Js be the collection of 2 intervals of length 4. T2-k+= that are evenly shifted by 11/2 T2 i.e.
Definition 2 (Seeded intervals). Let J Cs logg
Th = ((((i-())))) ((l(i-))22-) [(i - 1)72 + T2 T2-k+171 i= 1,.,..,,?? 2 13.
The overall collection of seeded intervals is denoted J L Jk
3

---

With the CUSUM statistics and the seeded intervals as building blocks, we are now ready to present our multivariate non-parann non-paramttric seeded change point detection algorithm.
Algorithm 1 Multivariate non-parametric Seeded Binary Segmentation. MNSBS (,,,
INPUT: Sample {Xt]t=s C IP,, collection of seeded intervals J, tuning parameter T > 0 and bandwidth h > 0.
initialization: If (8. e) = (0, n]. set S ++ 2 and set - log(T)
for I = (a. []] J do
if (a, 3] S (s, and 3 - a > 2p then
by - arg max maxxtpstss0p Fla.91 L2
0I F Ll
else
0I + -1
end if
end for
M = {I: : ag > T}
if M + 0 then
I' arg me [ Su (bz-)
MNSBS( (s. bz- J.J,),JTT T, MNSBS( + +1, end OUTPUT:
The set of estimated change points S.
Algorithm presents methodological approach to addressing the problem of estimating multiple change points in multivariate time series data. At its core, the algorithm leverages the strength of seeded intervals, forming a multi-scale search mechanism. To identify potential change points, the method recursively employs the CUSUM statistics. For the functionality of the algorithm, specific inputs are required. These include the observed data set, represented as XIT the seeded intervals denoted by J, the bandwidth that crucial for constructing the CUSUM statistics, and a threshold, which is instrumental in change point detection. We provide theoretical and numerical guidance for tuning parameters in Sections and respectively.
Delving deeper into the architecture of Algorithm D it becomes evident that the SBS functions as its foundational framework, while the nonparametric version of the CUSUM statisties acts as its functional units. The design of this algorithm is particularly tailored given its inclination toward nonparametric detection and its ability to identify multiple change points. The SBS is. in essence, an advanced version of the moving- window scanning technique. Its distinctive characteristic is its adaptability in handling the challenges posed by multiple change points that exhibit unpredictable spacing. Instead of being confined to a fixed window width, the SBS introduces versatility by incorporating a range of window width options. Each of these widths methodically applied during moving-window window scan.
Based on the preliminary estimators ( k= provided by Algorithm I we further develop a refine- ment procedure to enhance the localization accuracy. To be more specific, let
9
1
9 10 M++1 +
1
S
nh- +
10" The
and
ex =
10" T
(4)
10
Then, the preliminary estimators (me k= and x h produce an estimator of 66 as:
R
7k+11 T/
7
7mk f)-
Rk
m7++1 - 7/6 (5ie m
(fi+1
7 ( - 716
Tr. 1+1
im The +1
1.2
m 1 M/--1
(5)
77h ( - )

---

We then propose the final change points estimators as
xJ arg min (n),
(6)
Se <npe
where
ca
Qu(1) =
Σ
Pr,t, Ax Me +
Σ 1=
1
with h1 = CR2 21/r and F(s,e),ti = c-8 Di.e1 Fin, for integers e > 5.
If the initial change point estimators consistent, ie. (3) holds with probability tending to then the interval (fk-1, 1))+1) is anticipated to contain merely one undetected change point. By conservatively trimming this interval to ex), can safely any change points previously detected within 1)) Consequently, the trimmed interval (sk: ex) contains only true change point ?e with high probability. Due to the same reason, our choice of weight in Equation 4 1/10, is convenient choice. In general, any constant weight between 0 and 1/2 would suffice. Inspired by Padilla et al. (2021). who proposed use O (Kx) as an optimal bandwidth in the context of Lipschitz
densities, adopt h1 O
(((()))) (*)/") the bandwidth for our kernel density estimator. This choice
incorporates the broader scope of our work, which studies a more general degree of smoothness. Notably, if the underlying density functions strictly adhere to the Lipschitz criterion and our bandwidth selection aligns with that recommended by Padilla et al. (202I). We would like to emphasize that while the procedure proposed by Padilla et al. (2021) required knowledge of the population quantities Kg. our approach is adaptive as we provide data-driven methods to estimate accurately.
With our newly proposed estimators, in Theorem 2 we derive an improved error bound for the refined {fk )ix)... We also study the limiting distributions will discuss the theoretically justified rates and
estimators {īi 1 over the preliminary estimators
of the refined estimators. Section 444 and Section 5
practical choices of tuning parameters, respectively.
The computational complexity of Algorithm log") is due to the computational cost of the SBS, and "Kernel" stands for the computa- tional cost of numerical computation of the 12-norm of the CUSUM statistics based on the kernel function evaluated at each time point. The dependence on the dimension is only through the evalua- tion of the kernel function. The computational complexity of the final estimators (including estimating R3 's is ((T-Kenel)) Therefore, the overall cost for deriving (Th k= is O log TA -Kernel).
is ((Tg)) Kernel), where
4 Consistent estimation and limiting distributions
To establish the theoretical guarantees of our estimators, we first state conditions needed for the kernel function K(-).
Assumption 2 (The kernel function). Assume that the kernel function K(-) has compact support and satisfies the following additional conditions.
a. For the Hölder smooth parameter r in Assumption 74. assume that K(() is adaptive to H. HL, RP), i.e. for any f E H. HL, II), it holds that
sup WERF
h PK
1 -) (()) ((x)) [(x)
Čk",
h
12
for some absolute constant Ö and tuning parameter h > 0.
b. The class of functions Fx = [K(s - -//h TRP + R.+, >> 0} is separable in Tro((R)) (RP) and a uniformly bounded VC-class; Lee. there exist constants A, v > 0 such that for any probability
measure Q on BpP and any E (0, [[K]] it holds that N(Fr, Iz(Q), u) S (A[l(C]le..,))
where N VFFK, L2(Q), u) denotes the u-covering number of the metric space L2())..
Let [r] and it holds that Jo gm- sup [I(x)) m dt < <0, Sec K(())]|2||| de S Cx.
where Cx > 0 is an absolute constant.
5

---

Assumption iis standard condition in the non- parametric literature (e.g. Giné & Guillou] 1999 2001 Sriperumbudur & Steiwwart 2012 Kim al. [011 Padilla et al. 202T) and holds for various kernels, such as the Triweight, Hpanechnikov and Gaussian kernels, which are considered in Section
4.1 Consistency of preliminary estimators
To establish the consistency of the preliminary estimators outputted by Algorithm impose the following signal-to-noise ratio condition.
Assumption (Signal-to-noise ratio). Assume there exists an arbitrarily slow diverging sequence YT > 0 such that
A > YT log
We note that Assumption Jis mild condition, it allows both the jump size to vanish asymptoti- cally and/or the spacing A between change points to be much smaller than T. The consistency of Algorithm is established in the following theorem.
Theorem 1. Suppose Assumptions
/ and 3 hold. Let (7i R be the estimated change points (TTp/( (TT and h =cTT CCTT-1/(2r+
returned by Algorithm with tuning parameters
for sufficiently large constants Cb. Cr > 0. Then
P K = K. IM Th S C. TT log(T), L...k,..,,KK ...
-1-3CpxTT1,
where C. and Cp.xc positive constants only depending the kernel and the dimension p.
4.2 Refined estimators and their limiting distributions
To develop refined estimators based on the preliminary estimators and study their limiting distributions, we would need to require slightly stronger signal-to-noise ratio condition below.
Assumption 4 (Signal-to-noise ratio for inference). Assume that there exists an arbitrarily slow diverging sequence TT > 0 such that
+ Pr log (T) ,
Assumption is slightly stronger than Assumption This is because refined estimators based on a sequence of random endpoints, i.c the preliminary estimators. This brings theoretical challenges in deriving limiting distributions and estimating the long-run variances. It is worth noting that similar phenomenon has been observed in the study on conducted by Xu, Wang, Zhao & Yu (2022).
Theorem 2. Suppose that Assumptions I 2 and 3 hold. Let {ik
Ji Jk=1 be the refined change point
estimators defined in Section
3 with the preliminary estimators {Tı 1 returned by Algorithm
the intervals {ss ex) k= defined in (4), and Ra defined as in 5 The following holds:
)
a. -onvvannihing regime) Suppose the jump size at the change point my satisfies lime- Kk + 2k
for some absolute constant @k >> 0. Then, as T oo, it holds that Me Tpp Op(1) and that
(Tk
)
22 D
arg min Pa (F)
FEZ
where (7
El=7+1 2(Fnett.ng Smn te * Khng (Son S Kha)
+ + Smn) * Kha 12
F < 0;
F
Σi= 2(Fm tt.12 Smatt # Khay (frer - fyr) * Khalen + +ill(mm - fmn) ) * Kho 112 Fig
F > 0.
1/r
Here denotes convolution and fo2 Cus Cucki for some absolute constant Cug. > 0. b. (Vanishing regime) Suppose the jump size at the change point yi satisfies lim Ks Then,
as T + oo, it holds that iīj Tue = (k, mp
(k-pp=--pPp -2-p/r and that
2+2 2
TT TR) TH)
arg min (o (k)
(7)
FEZ

---

where h2 Cas b/ 3/r for some absolute constant Cuu > 0. Here
7 < 0, Bi(-r),
B(T)
O,
7 = 0,
B2)
F > 0,
with (r) and B2(r) being two independent standard Brownian motions, and
2-2
T
2:o (k:
lim T
Ki
Var
Σ
(Feh, - Se Kha (Srn - Smutz) * kn)
(8)
T
Theorem 2 considers vanishing and non-vanishing regimes of the jump sizes, The upper bounds on the localization enror in both regimes can be written as
F. E+2 Op(1).
max ISAS
7
Therefore, when the Hölder smoothness parameter r = 1, our final estimator {7) attains the minimax optimal convergence rate developed in Lemma 3 by Padilla a et al. (2021). Furthermore, when 1, our resulting rate is sharper than that Theorem in Padilla et al. (2021), as we are able to remove method achieve optimal rates the logarithmic factors from the upper bound. Additionally, with choices of tuning parameters that do not depend on the unknown jump sizes Ki-
Theorem 2 summarizes the limiting distributions of the refined estimators { In the non- vanishing case, the resulting limiting distribution can be approximated by two-sided random walk and the change points can be accurately estimated within constant error rate. In contrast, in the vanishing regime, central limit theorem under mixing conditions leads to a two-sided Brownian motion distribution in the limit, which quantifies the asymptotic uncertainty of (7i { enabling inference on change point locations, and allowing for the construction of confidence intervals.
4.3 Consistent long-run variance estimation
To obtain valid confidence intervals for change points using the limiting distributions in Theorem 2b.. it is crucial to access robust estimators for the long-run (asymptotic) variances {o3 ( )== defined in (8) We propose a block-type long-run variance estimator in Algorithm 2 to fulfill this task and demonstrate its consistency in the following theorem.
Theorem 3. Suppose Assumptions / and hold. Let { (k) K
1= be the population long-run vari-
ance defined in (8) and -2 (k) the output of Algorithm with R = O(T(ot)) / Di (/(r) (2r)+3/2 Then holdy that
22 (2) 52 (k)
F as +
max
4.4 Discussions on MISBSS
Tuning parameters. Our procedure comprises three steps: (1) preliminary estimation, (2) local refinement, and (3) confidence interval construction with three key tuning parameters. For step (1), use kernel density estimator with bandwidth h I1(((2++p)) which follows from the classical non-parametric literature (e.g. Yu 1993 Tsybakov 2009). The threshold tuming parameter 7 set to a high-pu probbbbbbbbiltyy upper bound the CUSUM statistics when there is change point, which reflects the requirement on the signal-to-noise ratio detailed in Assumption 3 For refined estimation in step (2) and long-run variance estimation in step (3), we set the bandwidth parameter h; choice of bandwidth inspired by the minimax rate-optimal bandwidth used in Padilla et al. (2021).
=1/r This R
Comparison with Padilla et al. (2021). Our main contribution is deriving the limiting distribution of multivariate non-paramttric change point estimators. This problem has not been formally studied in the existing literature. Additionally, our Hölder assumption is more general than the Lipschitz assumption used in Padilla al.] (202T). Our Assumption D1 specifies changes through the -grmorm of probability density functions, which is weaker assumption than the L -nom used in Padilla

---

Algorithm 2 Long-run variance estimators
INPUT: (x1)==1 (f (Rw) {(5ks (x))) and tuning parameter R E N
for :.....K. do
Let h1 = cz
for f E {
1 do
Yi = R
Fi,hi - fe * Khi (fin - fints) * Khi d 12
end for S [-
for r E (.....,
R] do
S,
{9k + (r 1)...... : :k +r5 -
end for
220 (k)
R a Eies,
1 Σ
EieE x))²
end for OUtpuT: (k))
et al. (2021). Furthermore, our assumptions allow for temporal dependence captured by a-mixing coefficients, whereas Padilla et al. 2021) assumed independent observations.
Comparison with existing literature in nonparametric, online, and inference change point.
In the nonparametric change point literature, different kernel-based methods are adopted for change point localisation and testing. In the offline setting, the penalized kernel least squares estimator, origi- nally introduced by Harchaoui & Cappé (207)), was explored by AAlot et al. (2012) for multivariate change point problems, and an oracle inequality was derived. An upper bound on the localization rate provided by this method was established Garreeu & Arlot (2018) and was computationally enhanced further in Celisse al. (2018). With focus on so-called running maximum partition strategy, Harchaoui et al. (2008) formulated kernel-based test statistic to ascertain the existence of change-point. a similar vein, Zou al. (2014) investigated problem where 8 out of sequences are anomalous and devised test statistic using the kernel maximum mean discrepancy.
In the online setting, Kifer al. (2004) introduces meta- mega--agooithmm comparing data from a "reference window" to current data using empirical measures, Desobry et al. (2005) detects shifts by comparing two descriptor sets from the signal's immediate past and future, using a dissimilarity metric resembling the Fisher ratio in Gaussian cases via a soft margin single-class SVM. Meanwhile, [iu et al. (2013) adopts density ratio estimation with a non-parann parameeriii Gaussian kernel model for change-poing detection, updating its parameters online through stochastic gradient descent.
The core methodology largely shared but with different goals and performance measurements regarding online and offline change point literature comparisons. How to conduct inference in the online change point context is also unclear.
Compared to the existing work, in this paper, we follow the suit using kernel-based CUSUM statistics but incorporate temporal dependence, which is rarely seen in the literature. Most importantly, we are unaware existing work on nonparametric change point inference, which is the main selling point of paper.
Most change point inference work focuses on niedddimensional parameters as well lacks tracking of many model parameters. Xu, Wang, Zhao & Yu (2022), in terms of style, is indeed the most closely related. But tackles high-dimensional linear regression, fundamentally distinet from our nonparametric density estimation.
5 Numerical Experiments
We refer to MNSBS as our final estimator, which is used for both localization and inference tasks. To evaluate localization performance, we compare our proposed method against four competitors MNP (Padilla et al. 2021). EMNCP (Matteson & James 2014), SBS (Cho & Fryzlewicz 2015) and DCBS (Cho 2016) across wide range of simulation settings, using corresponding R functions
in changepoints (Xu. Padilla, Wang & Li 2022), ecp (James al. 2019) and hábinseg (Cho

---

packages. However, to the best of our knowledge, no competitor is currently available for the inference task.
& Fryzlewicz 2018)
Tuning parameters. For MNSBS implementation, we use the Gaussian kernel and the false discovery rate control-based procedure Padilla et al. (202T) for selection. Preliminary estimators are set as h 2 x (1/T) while the second stage estimator has bandwidths respectively set as h = 0.05 and h1 = 2 x 1///r Selection of R = [maxf= fez - sx1)%/s with {(SE, cx)) is guided by Theorem 3 using {(*k, <x) ex)] from For the confidence interval construction, we use [R+]k=] k= and (a ]+)] estimate the required unknown quantities. We evaluate L2 based
3/5
statistics in Change point estimation and Long-run variance estimation using the Subregion-Adaye
Vegas Algorithm with maximum of 10 function evaluations.
Evaluation measurements For given set of true change points C = (m= ) K+1 assess the accuracy
of the estimator = {he J with nj 1 and f1 = T + 1f. we report (1) Misestimation rate: the proportion of misestimating K and (2) Scaled Hausdorff distance: dh(C, C), defined by du(č,C) = f/2 max max (max,ee minyeel a - y/}. max minzee { x - y(}}-
The performance of our change point inference is measured by the coverage of M|, defined as covers (1 a) for significance level E (0, 1). For, k = I,..., K,
cover (1 - aa) =1 y/u E mh + pp/r22 The Fu (u(1-a/2)
(a(a22)
/p/r+2
Kg
R
with (a/2) and Tr (1- a/2) are the a/2 and 1 1 a/2 empirical quantiles of the simulated limiting distribution given in (7). Rg is defined in (5), and k = I....,.. K.
5.1 Localization
We consider three different scenarios with two equally spaced change points. For each scenario, we
set 2, and vary T {150, 300} and p {3, 5}. Moreover, we consider {Y, [[[3]] < < / X, 1'
C Rº with X, = 0.3X,,1 tt.
Scenario 1 (S1) Let Z, = μ E TR, where , = 0 f j E (1,....[[//2) and j4j otherwise. Let {4} be i.i.d. N (oppp Ip).
Scenario 2 (S2) Let /u = 1} 1.5x x 1. Zu 0} = -1.5 x 1p. where { are i.i.d. (emnoulli(0.5) random variables. Let {et] be i...d. (o Ip).
Scenario 3 (S3) Let Z, 0.3Z,-1 + 0.5 x 1p + where { c R' and {et C R' are mutually
independent. They are i.i.d. with entries independently follow Unif(- /3, v3) and the standardized P3reto 1), respectively.
S1-S3 encompass a variety of simulation settings, including the same type of distributions, changed mean and constant covariance in S1; a mixture of distributions in S2; and change between light-tailed and heavy-tailed distributions in S3. We conduct 200 repetitions of each experiment and present the results for localization in Figure [ Our proposed method, MNSBS, generally outperforms all other methods in all scenarios, except for S2, where ECP performs better. However, we observe that MNSBS achieves comparable performance to ECP for large T in S2.
5.2 Inference
In this section, we focus solely on analyzing the limiting distribution obtained in Theorem bb. which pertains to the vanishing regime. We explain this from two different perspectives. Firstly, in the non-vanishing regime (Theorem 2 a.), the localization error is at the order of O(1). As a result, the construction of confidence intervals, which is direct application of the limiting distribution, is of little demand with such estimation results. Secondly, since the localization error is only at the order of O(),, the universality cannot come into play produce useful limiting distribution.
We consider the process (Y, = 1[T722] < t < T Th + X,) with X, = 0..3X,1 + ez, Here, Ip and { are i.i.d. (O,,, 1,). We vary T {100, 200, 300} and p E {2, 3), and observe
'The Subregion AAdatiive Vegas Algorithm is available in R package cubature Narasimhan al. 2022

---

T-150
T+500
Tw150
T-300
T=150
T-300
p
p
0-33
p-5
1.02
p
p
p
0.75
0.00
0.33
T-150
T-300
T=150
T-300
T-150
T-300
p.3
0.25
0.22
0.17
0.27
0:06
Figure 1: From top to bottom: Misestimation rate of the number of change point K; Scaled Hausdorff distances. From left to right: Scenarios S1-S3. Different colors represent different methods, ordered as MNSBS, NMP, ECP, SBS, DCBS.
Table I: Results for change point inference.
0.01
0.05 width x))
cover(1 (1- a)
width(1 a)
cover(1 a)
p=
0.864
17.613 (6.712)
0.812
14.005 (5.639)
200
0.904
22.940 (7.740)
0.838
18.407 (6.541)
300
0.993
26.144 (9.027)
0.961
20.902 (5.936)
3
0.903
15,439 (5.792)
0.847
11.153 (4.361)
200
0.966
20,108 (7.009)
0.949
13,920 (5,293)
300
0.981
22.395 (6.904)
0.955
15.376 (4.763)
that our localization results are robust to the bandwidth parameters, yet sensitive to the smoothness parameter r. We thus r 1000 in our simulations, the density function of multivariate normal distribution belongs to the Hölder function class with = oc. Table I shows that our proposed inference procedure produces good coverage in the considered setting.
Conclusion
We tackle the problem of change point detection for short range dependent multivariate non- oonpaaamettric data, which has not been studied in the literature. Our two-stage algorithm MNSBS can consistently estimate the change points in stage one, novelty in the literature. Then, we derived limiting distributions of change point estimators for inference in stage two, a first in the literature.
Our theoretical analysis reveals multiple challenging and interesting directions for future exploration. Relaxing the assumption x may be of interest. In addition, in Theorem a we can see the limiting distribution is function of the data-generatin mechanisms, lacking universality, therefore deriving practical method to derive the limiting distributions in the vanishinn regime may be interesting.
10

---

Acknowledgments and Disclosure of Funding
HX was supported by Swiss National Science Foundation Postdoc Mobility fellowship. OHMM
partially funded by NSF DMS-2015489, YY's research is partially funded by EPSRC EP/VO13432/1.
References
Abadi, M. (2004), 'Sharp error terms and neccessary conditions for exponential hitting times in mixing processes', The Annals of Probability 32(1A), 243 264.
Arlot, S., Celisse, A. & Harchaoui, Z. (2012), 'A kernel multiple change-point algorithm via model selection', arXiv preprint arXiv: /202.33/022.8777
Arlot, S., Celisse, & Harchaoui, Z. (2019), 'A kernel multiple change-poing algorithm via model selection', Journal of machine learning research 20(162).
Aue, A., Hörmann, S., Horváth, L. & Reimherr, M. (2009), 'Break detection in the covariance structure of multivariate time series models', The Annals of Statistics 37(6B), 4046-4087.
Azhar, M. A. R., Nugroho, H. A. & Wibirama, S. (2021), The study of multivariable autoregression methods to forecast infectious diseases, in '2021 IEEE 5th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE)', IEEE, pp. 83-88.
Boussama, F., Fuchs, F. & Stelzer, R. (2011), 'Sttaiinarityy and geometric ergodicity of bekk multi- variate garch models', Stochastic Processes and their Applications 121(10), 2331-2360.
Cai, X., Wang, X., Eichi, H. R., Ongur, D., Dixon, L., Baker, J. T., Onnela, J.-P. & Valeri, L. (2022), 'State space model multiple imputation for missing data in non-stationary multivariate time series with application in digital psychiatry'. arXiv preprint 22t1::2220..444333
Celisse, A., Marot, G., Pierre-Jean, M. Rigaill, G. (2018), 'New efficient algorithms for multiple change-poin poontte-pinttoooo detection with reproducing kernels', Computational Statistics & Data Analysis 128, 200-220.
Chan, K.-S. & Tong, H. (2001), Chaos: a statistical perspective, Springer Science & Business Media.
Cho. H. (2016). 'Change-point detection in panel data via double cusum statistic', Electronic Journal of Statistics 10(2), 2000-2038.
Cho, H. & Fryzlewicz, (2015), Multiple-change-pone detection for high dimensional time series via sparsified binary segmentation', Journal of the Royal Statistical Society: Series B (Statistical Methodology) 77(2), 475-507.
Cho. H. & Fryzlewicz, P. (2018), hdbinseg: Change- Analysis of Diige-Dimesioonl Time Series via Binary Segmentation. package version 1.0.1.
URL:
Corbella, S. & Stretch, D. D. (2012), 'Predicting coastal erosion trends using non-stationary statisties and process-based models', Coastal engineering 70, 40-49.
Desobry, F., Davy, M. & Doncarli, C. (2005), 'An online kernel change detection algorithm', IEEE Transactions on Signal Processing 53(8), 2961-2974.
Doukhan, P. (1994), 'Mixing; properties and examples. lect', Notes in Statisit 85.
Fan, J. & Yao, Q. (2003), Nonlinear time series: nonparametric and parametric methods, Vol. 20, Springer.
Fan, J Yao, Q. (2008), Nonlinear time series: nonparametric and parametric methods, Springer Science & Business Media.
Frolov, N., Maksimenko, Hramov, A. (2020), 'Revealing a multiplex brain network through the
analysis of recurrences'. Chaos: An Interdisciplinary Journal of Nonlinear Science 30 12), 108.

---

Fryzlewicz, P. (2014), 'Wild binary segmentation for multiple change-poid detection', The Annals of Statistics 42(6), 2243-2281.
Garreau, D. & Arlot, S. (2018), "Consistent change-point detection with kernels..
Giné, E. & Guillou, A. (1999), 'Laws of the iterated logarithm for censored data', The Annals of Probability 27),), 2042-2067.
Giné, E. & Guillou, A. (2001), 'On consisteney of kernel density estimators for randomly censored data: rates holding uniformly over adaptive intervals', Annales de I'IHP Probabilités et statistiques 37(4), 503-522.
Gorrostieta, C., Ombao, H. Von Sachs, R. (2019), 'Time-dependent dual-frequency coherence in multivariate non- con-stationary time series', Journal of Time Series Analysis 40(1), 3-22.
Han, F. & Wu, W. B. (2023), Probability inequalities for high-dimensional time series under triangular array framework, in "Springer Handbook of Engineering Statistics', Springer, pp. 849- 863.
Harchaoui, Z. & Cappé, O. (2007), Retrospective mutiple change-pone point estimation with kernlls, in '2007 IEEE/SP 14th Workshop on Statistical Signal Processing' IEEE, pp. 768-772.
Harchaoui, Z., Moulines, E. & Bach, F. (2008), 'Kernel change-pone analysis', Advances in neural information processing systems 21.
Heo, T. & Manuel, L. (2022), 'Greedy copula segmentation of multivariate non-stationary time series for climate change adaptation', Progress in Disaster Science 14, 100221.
Herzel, S., Stärică, C. & Tütüncti, R. (2002), A non-stationary multivariate model for financial returns, Chalmers University of Technology.
James, N. A., Zhang, W. & Matteson, D. S. (2019), ecp: An R Package for Nonparametric Multiple Change Point Analysis of Multivariate Data. R package version 3.1.2. URL: https://cran.rrr hwwp::n.--poeeeetoogaaaaaaaeeee.
Kifer, D., Ben-David, S. & Gehrke. J. (2004), Detecting change in data streams, in 'VLDB', Vol. 4, Toronto, Canada, pp. 180-191.
Kim, J., Shin, J., Rinaldo, A. & Wasserman, L. (2019), Uniform convergence rate of the kernel density estimator adaptive to intrinsic volume dimension, in 'International Conference on Machine Learning', PMLR, pp. 3398-3407.
Kirch, (2006), Resampling methods for the change analysis of dependent data, PhD thesis, Universität zu Köln.
Kovács, S., Li, H., Bühlmann, & Munk, A. (2020), 'Seeded binary segmentation: general methodology for fast and optimal change point detection' arXiv preprint arXiv:2002.06633.
Kunitomo, N. & Sato, S. (2021), 'A fobust-filtering method for noisy non-stationary multivariate time series with econometric applications', Japanese Journal of Statistics and Data Science 4(1), 373-410.
Li, S., Xie, Y., Dai, H. & Song, I.. (2019), 'Scan b-statistic for kernel change-point detection', Sequential Analysis 38(4), 503-544.
Liebscher, E. (2005), 'Towards a unified approach for proving geometric ergodicity and mixing properties of nonlinear autoregressive processes',
"ncesses",,Jurrae JJornnll Time Series Analysis 26(5), 689.
Liu, S., Yamada, M., Collier, N. & Sugiyama, M. (2013), 'Change-point detection in time-series data by relative density-ratio estimation' Neural Networks 43, 72-83.
Matteson, D. & James, (2014), nonparametric approach for multiple change point analysis of multivariate data", Journal of the American Statistical Association 109(505), 334-345.
12

---

Merlevède, F., Peligrad. M. & Rio, E. (2012), 'Bernstein inequality and moderate deviations under strong mixing conditions', arXiv preprint arXiv:1202.4777.
Merlevède, F., Peligrad, M., Rio, E. et al. (2009), 'Bernstein inequality and moderate deviations under strong mixing conditions', High dimensional probability V: the Luminy volume 5, 273-292.
Molenaar, P. Sinclair, K. O., Rovine, M. J., Ram, N. & Corneal, S. E. (2009), 'Analyzing develop- mental processes on an individual level using nonstationary time series modeling.", Developmental psychology 45(1), 260.
Narasimhan, B., Johnson, S. G., Hahn, T., Bouvier, A. & Kiču, K. (2022), cubature: Adaptive Multivariate Integration over Hypercubes. R package version 2.0.4.5.
URL: https://CRAN. R-p R-woo--eroorreeeeeeeeeeee hrapeettoorgpcckgeeeuuuuuttr
Nguyen, H. M., Turk, P. J. McWilliams,
(2021), "Forecasting covid-19 hospital census: A multivariate time-series model based on local infection incidence', " Public Health and Surveillance 7(8), e28195.
Padilla, C. M. M., Wang, D., Zhao, Z. Yu, Y. (2022), 'Change-poii detection for sparse and dense functional data in general dimensions', arXiv preprint arXiv:2205.09225..
Padilla, O. H. M., Yu, Y., Wang, D. Rinaldo, A. (2021), 'Optimal nonparametric multivariate change point detection and localization', IEEE Transactions on Information Theory.
Schmitt, T. A., Chetalova, D., Schäfer, R. & Guhr, T. (2013), 'Non-stationarity in financial time
series: Generic features and tail behavior', EPL (Europhysics Letters) 103(5), 58003.
Sriperumbudur, Steinwart, (2012), Consistency and rates for clustering with dbscan, in 'Artificial Intelligence and Statisties', PMLR, pp. 1090-1098.
Tsybakov, A. B. (2009), Introduction Nonparametric Estimation, Springer series in statistics, Springer, Dordrecht.
van der Vaart, A. W. & Wellner, J. A. (1996), Weak Convergence and Empirical Processes: With Applications to Statistics, Springer New York, New York, NY. URL:
Venkatraman, E. S. (1992), Consistency results in multiple change-poing problems, Stanford Univer- sity.
Wang, D., Yu, Y. & Rinaldo, A. (2020), 'Univariate mean change point detection: Penalization, cusum and optimality', Electronic Journal of Statistics 14(1), 1917-1961.
Wolkovich, E. & Donahue, M. J. (2021), 'How phenological tracking shapes species and communities in non stationat environments' Biological Reviews 96(6), 2810-2827.
Xu, H., Padilla, O., Wang, D. & Li, M. (2022), changepoints: A Collection of Change-Point Detection Methods. R package version 1.1.0. URL:
Xu, H., Wang, D., Zhao, Z. & Yu, Y. (2022), 'Change point inference in high-dimensional regression models under temporal dependence' arXiv preprint arXiv:2207.1245.
Yu, B. (1993), 'Density estimation in the Lo norm for dependent data with applications to the gibbs sampler', The Annals of Statistics pp. 711-735.
Zou, S., Liang. Y., Poor, H. V. & Shi. X. (2014), 'Nonparametric detection of anomalous data via kernel mean embedding', arXiv preprint 1005 2492
13

---

A Additional simulation results
A.1 Robustness to Kernel Selection and Bandwidth Parameters
In this subsection, provide additional simulation results in Table 2 to show that proposed localization method, i.e. MNSBS, is robust against both the choice of Kernel functions and the choice of bandwidth parameters.
We consider the setting with T = 150 and p = of Scenario 1. addition to the Gaussian kernel used in the previous numeric experiments, we consider the Epanechnikov kernel and the Triweight kernel. We also let the bandwidth h ca x (1/T))/((r+p) with 2 and сą E {1, 5, 10, 20). Out of the 500 iterations for each case, the table below reports the proportion of the number change points K and the averaged localization errors.
Table Additional localization results of Scenario 1.
T 15000, = and
Kernll
Ch
CA =
Ch 5
Cp. 10
Cj. 20
Propotion of times K + K
Gaussian
0.138
0.070
0.066
0.070
0.068
Epanechnikov
0.748
0.184
0.070
0.070
0.064
Triweight
0.280
0.082
0.068
0.066
0.060
Average (standard deviation) of dh
Gaussian Epanechnikov Triweight
0.038(0.050)
0.029(0.043)
0.025(0.039)
0.026(0.040)
0.026(0.038)
0.118(0.045)
0.012(0.035) 0.037(0.049)
0.011(0.035)
0.010(0.035)
0.053(0.057)
0.017(0.038)
0.010(0.034)
0.010(0.035)
0.010(0.034)
A.2 Runtime and localization for Independent Data
Examination of Scenario 1 with independent data. We examined Scenario where p 3, n is in the set (150, 300}, and X, is i.i.d. distributed as N (0p, Ip). Our results indicate that MNSBS excels in change point localization. The refinement process further enhances its performance. Refer to Table specifics.
Runtime Comparison.
We benchmarked the runtime of our method against others. The tests were conducted on machine powered by an Apple M2 chip with an 8-core CPU. The parameters were set at p = 3 and n in the set {150, 300) for the independent setting. Our method performs comparably at 150. However, is slower at n = 300, attributed to the computational demands of CUSUM. Detailed findings are presented in Table
Table 3: Runtime and localization results of Scenario with independent data.
300
p
p
AVERAGE (STANDARD DEVIATIOX) OF RUNTIME (IN SECONDS)
MNSRS(NIITAAL & REJINER)
1.130 (71)
13.134 (2 .006)
NMP
(1.742)
ECP
05305)
0.645 140)
SBS
0,679 (0.033)
0.656 (0.040)
DCBS
0.366 092)
1,354 (3.134)
PROPOTION OF TIMES #
MNSBS(INTTTA)
0.000
6,085
MNSBSQLEPNED)
0.000
6.005
NMP
ECP
0.000
0.040
SBS
0.445
0.015
DCBS
0.075
0.065
AVERAGE (STANDARU DEVIATIUN)
(INSBS/INITAL)
0.030 (0.01.5
0.009 (0.012)
MNNDS(REENNE))
0.006 (0.011)
0.985 (0.011)
NMP
0.016
SBS
0.158 (0.157)
0.012 (0.041)
DCBS
0.021 (0.057)
0.011 (0.022)
14

---

B Real data application
We applied our proposed change point inference procedure analyze stock price date which consisted of daily adjusted close price of the 3 major stock market indices (S&P 500, Dow Jones and NASDAQ) from Jan-01-2021 Jan- 19-2023. After removing missing values and standardizing the 3.
raw data, the sample size was n 515 and the dimension
We localized estimated change points and performed inference based on them; results summa- rized in Table 4 We also implemented the NMP and ECP methods on the same dataset, the estimated change points being presented below. Except for the time point Aug-24-2022 estimated by ECP, all other estimated change points were located in the constructed 99% confidence intervals by our proposed method.
The transformed real data is illustrated the figure below. These data correspond to the daily adjusted close price, from /an-01-202) Jan-19, 19-2023, of the major stock market indices, S&P 500, Dow Jones and NASDAQ. Moreover, in Table we present the estimated change point by our proposed method MNSBS the data before mentioned, together with their respective inference.
sandrdaed Saly drocpfice
ma
100
200
300
400
500
Figure 2: Plot of the standardized daily close price, from Jan-01-2021 to Jan 19-2023, of the major stock market indices.
Table 4: Confidence intervals constructed for change point locations the Real data example.
0.01
0.05
LOWER BOUND
UPPER BOUND
LOWER BOUND
UPPER BOUND
APRIL-07-2021
APRH--01-2021
APRIL-12-2021
APRI--05-2021
-09-211
JUNE-30-2021
JUNE-23-2021
Jul.Y-9--0021
JUNE-25-2021
JULY-07-2021
OcT-19-2021
OCT-12-2021
OcT-26-2021
DCT-14-2021
OCT-22-2021
JAN-18-2022
JAN-12-2022
JAN-21-2022
JAN-13-2022
JAN-20-2022
APR12-25-2022
APRIL-20-2022
APRIL-28-2022
APRIL-21-2022
APRIL-27-2022
OcT-27-2022
OcT-24-2022
Nov-01-2022
OcT-25-2022
OCT-31-2022
The result of the implementation of NMP and ECP methods on the same dataset are (April-01-200
July-01-2021, Oct-19-2021, Jan- 14-222, April-21-2022, Oct-26-2022) and (April-08-200 June-25-2021. Oct-18-202) Jan-18-2022, April-28-2022. 2ug-24-2022, Oct-27-2022) respectively.
"The stock price data downloaded from http:://rreeddetoooiidddddd
15

---

C
a-mixing coefficients
This paper focuses multivariate time series that exhibit a-mixing behavior with exponential decay coefficients. This condition is denoted Assumption P.. While the constant 2c is present in the exponent of the exponential function, plays non-essential role in our theoretical framework. We include it solely for the sake of convenience during verification.
The a-mixing condition with exponential decay specified in Assumption 1 commonly held assumption in time series analysis. A broad spectrum of multivariate time series satisfies this condition, including linear/nonlinear VAR models (e.g. [iebscher (2005) comprehensive class of GARCH models [c.g. [Boussama et al. (201T)), and various Markov processes [e.g. Chan & Tong (2001)) To further elaborate, consider the p dimensional stationary VAR(I) model:
X AX,-
where A is the pxp transition matrix whose spectral norm satisfying |||||| E (0, 1) and the innovations 6 are i.i.d. Gaussian vectors. Denote cov (X1) ) and let Amaa and Xmin be the largest and smallest eigenvalues of Σ. Then by Theorem 3.1 in Han & Wu (2023). we have that for any k 2 0, the
o-mixing coefficient of the time series X, satisfying
Amax
c S Cloo(((/AAD))
Amin
where C O is constant depending only ('log(1|[A) corresponds to the constant 2c in Assumption De. Essentially, Assumption [ is useful unlock several technical tools under temporal dependence, which include a Bernstein's 's inequality Merlevède et al. (2012), moment inequality [see Proposition 2.5 in Fan Yao (2003)1, maximal inequalities (see Section G. and central limit theorem (see Section G.2). For instance, utilize the moment inequality to bound the autocovariances of a dependence process with all lags by a- mixing coefficients, thereby demonstrating the existence the long-run variance, which is the sum of all the autocovariances.
Amaxx Amin- In this example, the constant
16

---

D Proof of Theorem 1
In this section, we present the proof of theorem Theorem |
Proof of Theorem
! For any e C (0, T,, let
e
(x) (x) (0))=1 V (e st Σ
fi(x)
s
Σ
f((x), x X.
V (e s)(c - 1=t11
1=++1
For any F ( p). we consider
A((s,c),p.))
max
sup HERP
F (2) (x))
(x) =>}
t=s+p++
F+NN
F+N
T-F max sup N=p acRP
Σ f1(z) SA
B(F,
(a)())
VN
1=111
t=7+1
Σ f((z) sa).
max sup N= SERP
Σ
Fin(z)
N+1
N+1
From Algorithm 1 we have that
log(T)
p
hp
Therefore, Proposition
2 imply that with
log T
2CIVD
2C2 2222 2222THT.).
A = C2 20
+
(9)
he
Vhe
for some diverging sequence Cx it holds that
p
pf ((A
x T2.
and,
P
B
Now, we notice that,
8
n
R
(2 1)
R
2h < 2(2
Ca
lng
)
1) S 2(2
[a
(og(7)
1) O(T)..
since n < 1 for any > O. In addition, there are K = O()) number of change points. In consequence, follows that
I A(I, for all 21 2 1 T
(10)
B(s, for (s,e]=Ie I E 21 7
(11)
1 P B(Te. p, X) for all S S k 21 21-1 T
(12)
The rest of the argument is made by assuming the events in equations (10) and 12 hold. By Remark I we have that on these events, it is satisfied that
(x Ft (x))le, A. t=stpt1
max
17

---

Denote
Ts = C log(T)
T
- kik
and
Ymax C' (Tg
T N=
where &. = minffi........ NK). [(nk
Since Ys is the desired localisation rate, by induction, it suffices to consider any generic interval e8 C (0, () that satisfies the following three conditions:
7m S - 7m S ... 7m-- S e S +m+
q 2 -1;
either mm T
Mm- T mm
either Mm++++1 e Σ T m++++1 or c - Mm++y S Y m+
Here -1 indicates that there is no change point contained in (8,e).
Denote
A 9-- T7e for k
,,...,K K and A min A...... AKtI).
Observe that by assumption 3
A
Y max
C Co)g(T)
[
4
Therefore, has to be the case that for any true change point 7/m E (0, T) either 1m s| < T. or
Pm 2 A T max 2 3 A. This means that min (1/m el, 1m - =|3 S Y, indicates that 7mm is a detected change point in the previous induction step, even if 1mm E el. We refer to Tmm E (s, e] as an undetected change point if min (iin S, 7m -e 2 A. To complete the induction step, suffices to show that MNSBS((s, el, h,
(i) will not detect any new change point in (s, el all the change points that interval have been previously detected, and (ii) will find a point D² in (s, e] such that 1m - D2' S Ym there exists at least one undetected change point in (s, e].
In order to accomplish this, we need the following series steps.
Step 1. We first observe that if 7/s E {TH k=1 is any change point in the functional time series, by Lemma 5 there exists a seeded interval I& (sk, es] containing exactly one change point 7i such that
9
min
8k. Pk m) 2
16 A, and max
Sk; ek m) S
10
Even more, notice that if m7 (s, is any undetected change point in (s, c). Then it must hold that
Tjk-1 T max-
Since Max = (log((T)T<>>>)) and by assumption we have that Y max < 10 1.A. Moreover,
U
8k I 10 (m 1)-1),, so that it holds that
8 77k-- 77--
10 (ms 7h T max 2 1---
and in consequence 8& 2 s. Similarly < Therefore
I (sk, en] es,
Step 2. Consider the collection of intervals (IA- = (sk, 6x]] in Step 1. In this step, it is shown that for each & E (I,...,K K}. holds that
Cy [(sh.e)) max ||
1ll C1 C.AA
(13)
1=8
for some sufficient small constant C1.
Let k E (1,..., KJ. By Step 1, In contains exactly one change point mkk- Since Js one-dimensional population time series and there is only one change point in Ij (8R, en), it holds that
fatt1 _.. fm * fm+t1 ffx
18

---

which implies. for S/. <t < 7k
Mow.ee V (ek - sk) (t- 8k) 1=8++1
6k -1
Σ fm V (ex - sx](es - -t)
1 5k
Σ fn
1=1-1
Ch
8- 8k
fm
V (ek 8kk - 1=
=(1 8k2 V (ex 81)(1 8x)
Ck -
Jm (T - -D [ex
t 8k
sx)(ex 1) Smz
888
- (ex m&b V (e* sx)(ex t) frat1
(1 - 8x) (ex 1)
1 - 8k
Im (Mk 1 (ex SE) - t) 5m
(ex
sk)
{ -
(ex
m1 V
(ek 8/ Sek 1) smat1
-(ew
V
3k
sm -IIx DV (Ek se)(ee - 1) sm
t - 8k
(ek ()x 8k))
(ex 176
(ex Sex - 1) fma+1
8k
t 8k Jmn -eex - -ma) (es - (x) 1) Stat1
=(ex mmk) |
(ek ()(x - sk)
=(ex 1
(fna - Smit)).
(ex ()(xx= - sk)
Similarly, for 7 St Ek
1 s((en
Ck -
(n sk)(fn -mutz)-
V (ex - sx) - sk)
Therefore,
t- kt (ex-ex](es- (ek The ] fn1t2),
8y <t< < mp:
M
(14)
(ex-8t)]( [[| ) (11 sk ) (frr Sm+t)),
TA S S Ck.
Since p = Ollog(T) (T)og ), by Assumption we have that
1
min 88,, Cx m)
P:
(15)
16
so that T) [stk + Px p}. Then, from 14 (15) and the fact that lee - skl < A and
Iyx - 8k A
|1 flnnee 11z2 =
ek UE
(ne st)llfm - Smitilla 2 voVA
4 Kk- (16)
(ex
-s)( sk)
Therefore, holds that
tmmg -0
|| [(sn.ca]
||,, 21 2/ eul llla
max
1=a++
2 211 ||:0 X
n/
3
N0² 4
/ Arss - X
19

---

where the first inequality follows from the fact that ? E [sk + P. pl, the second inequality follows from the good event in (TO) and Remark Next, we observe that by Assumption
2 and the last inequality follows from (T)..
log²
(T) VTV hP
T (Tg
2 VAKs, 1
4
and,
VTW ス T न...
In consequence, since Rą is a positive constant, by the upper bound À on Equation for sufficiently large T, it holds that
C2 VAn 2 A
Therefore,
(=
|1
C2 ARk. 1lla 2
max tP
2
Therefore Equation
13 holds with C1 2
In this step, it is shown that CBS((s,ee c). h r) can consistently detect or reject the existence of undetected change points within (s, c).
Step 3.
Suppose Ti [s is any undetected change point. Then by second half of Step
Is S (8, e). and moreover
( =P [ max [1 ts +p
|ll, 2 c1 C1 Ang >
ag
where the second inequality follows from Equation 13 and the last inequality follows from
Assumption and the choice of == C- log (T) hF
Therefore, M e * 0. since Iç E M
Suppose there does not exist any undetected change point in (s, e]. Then for any I = (a, 3 S (s, one of the following situations must hold,
(a) There is no change point within (ax, 3]:
there exists only one change point nk within (a, 3] and min(me - a, R, n/ S Y4
(c) there exist two change points Ths M441 within (a, []] and
7 Tx and B - Mk+1 Y
Observe that if (a) holds, then we have
max | a+p<t<s-p E.
S max a+p<t<s-p
I1
( 11 + =\
Cases (b) and (c) can be dealt with using similar arguments. We will only work (c) here. It follows that, in the good event in Equation (TO).
max a+p<teB-p
1l S max
|1 F(a,9)
L22 +1
(17)
actes
$ Ve - M
V SKR A
(18)
$ 22/Clog (T)
T
(19)
where the second inequality is followed by Lemma 7 Therefore in the good event in Equation (10).
for any I (a, 3] (s, e). it holds that
dr =
max tattp
8-0
Ill < 2l (oCCCloggTV)V))) (T) T +
Then,
2 2vC log (T) - T +1
log T 201VP
-2VClog (T) hp
+1 201
hv +
vhe + +CC2Vhh.
20

---

We observe that V log(?) ha = (0(1o(TT)122 Viz) Moreover,
VTh" = VT
2
1 T ( スデスデ
T
and given that,
2
2r +p
2(2r + p)
get,
VTh - o (T)
Therefore, by the choice of T, we will always correctly reject the existence of undetected change points, since
(T)
+AST.
Thus, by the choice of r, it holds that with sufficiently large constant Cr.
ar T T for all C (s, e].
(20)
As result, MNSBB((s,.),h..)) will correctly reject if (s, el contains undetected change points.
Step 4. Assume that there exists undetected change point "h E (s, e] such that
min mh - s, 1% - -e} = A. Then, M * 0. Let I be defined as in MNSBS ((s, c), h, r) with
I (a* "a"",99) 3.
To complete the induction, it suffices to show that, there exists a change point Yy (s, e] such that c} 2 A and lbg- 77l Yg. To this end, we consider the collection of change points of {fi .3-| We are to ensure that the assumptions of Lemma 12 are satisfied. In the
min(7} 5, 7l
following, À is used Lemma 12 Then Equation (68) and Equation (69) are directly consequence of
Equation 10 Equation (IT), Equation (2). By the narrowest of I"
I SA
and by Step with Iz = (8k, ex), it holds that
1
min 8k. Pk 113}
C4 2 cz
16
Therefore for all l E (k: min 8, - n73 2 c2
f= P max || t=a +p
F() t=
1=e max
L, 2 c1 Ca V
where the last inequality follows from Equation (13) Therefore 770 holds in Lemma 12 Equation (11 is a direct consequence of the choices that
Finally,
p log(7) (T) nhd
h CT and
Thus, all the conditions in Lemma 12 are met. So that, there exists change point m7 of (fiher., satisfying
m Me, Tl a²) >A,
(21)
and
1
1zz- -mal 5 max (Ca) k 2 SCa log(T)
+
Th12
Kik
-2
h"
<Clog(T)
T
Kg
-2
21

---

for sufficiently large constant C, where we have followed the same line of arguments as for the conclusion of (2). Observe that
i) The change points of [Se hez belong to (s, e] n (m and
ii) Equation 21b and (a", ,8"] E (s, e] imply that
min T], MJ
8) c T max.
As discussed in the argument before Step 1, this implies that 7 must be an undetected change point of hft
22

---

E Proof of Theorem 2
In this section, we present the proof of theorem Theorem
2+
Proof of Theorem 2 Uniform tightness of Kig Te Th- Here we show a.l and b.1. For this
purpose, we will follow series of steps. On step we rewrite (6) in order derive a uniform bound. Step 2 analyses the lower bound while Step 3 the upper bound.
Step Denote 7 TJk. Without loss of generality, suppose F 2 0. Since Tik 7) + r. defined
in is the minimizer of (n). it follows that
Qu + + - -xQ())) SO.
Let
(n)
Σ
|1 flowmll Knzlli +
FL,12 fimmee Khzllia
(22)
t=
t=
where,
Σ
Sú, [m
fi.
(23)
7k
Sk i=85+1
Ck
1]6s
(=1
Observe that,
Q' (mh Q7) (Q((T))
Qu( - + Q- +
(24)
2+ I < 1/K/ then there is nothing to show. So for the rest of the argument, for contradiction,
assume that
(25)
2+
K
Step 2: Finding lower bound. this step, we will find lower bound of the inequality 24 To
this end, we observe that,
1/ 7F
(" + + (m)) =
Σ
|e - ftanon) * Knolli. -
Σ
|| - f(anec)) * Khzlli.
1=
t=++1
Σ
[[ Khng -Sinneeu Khaal 1
t= +1
Σ
(f(se.m)) # Khg Simmeeu * Krg: Fi,ho S(me # Khig
(1
[on Ammealll. 211f(rn.ma) Khs flo + Fineer * Krs fon.eu 12
t=
no+F
2
(formyl
Krss - fimeeo Khyy Finu - Simmeul Khna) La
t= Th
1
2- TKk
Σ
[lformmal * Kis - flsamel + fimeer Krs fin.eell 12.
t=
m++F"
Σ
(Framm Krss - Fim.ea Khng Finr - Sine.eul Knad La
1=++1
23

---

We consider,
nK- 1 =
[e * Khsa fe + [nneea * Kiz2 S( 12,2 and,
Ok OK-F 1/2 =
(( Khag - Simmrea Khg; Fe, - [(mneea) Khg
From above, we have that,
1 1- TK. - 12. 2
Q' (7) --
We now analyze the order magnitude of term I1. Then, we get lower bound for the term -11 In fact /1, has upper bound of the form Op ++))+ where
that 11 Kiz Sn. lla, o(1) and || fm Khv . llz = o(1). For the term 12, we consider the random variable.
Y, ([ac+1mm) Kha - [me+ Khgg Pi,n2 -mm++1ee Khggr
K6g B( Smm-1 Khza 11
In order Lemma 3 need bound [[((])))) For this, first Cauchy Schwartz inequality,
B( (110m++11 fmn Kn, 11 ) - -Seas Kha 1122)
n2 BB Smrta * (h,2li,,))
then, by Minkowski's inequality,
[1(m ml - sma. * Khig 1|ا0 =11 / (Jm+1 - Smn)( 4) Knp (y)dy La
$ [f fna)(- y) 2) (v)
dy
L2z (Fout1 Sm y) fm+t1 Jm 21 1Kngllee
IKhn (hnlvildy)) )1
Ba
Therefore, by Assumption 2 we have
(1([m 5mn Krs 11==)===(((11 B - Smn * Khhg 11)22)
B 12, 8 Fni 1 Kha Khall 1222)
(1 [1 Smmllee 11 KKrall )
Smass Kh l1i2)
x2110(1 Fe, Sman * KNall22))
SCK.
for any t (Nk; ex). Moreover, we have that
(I( - E(Kn,( E(Kh,(x-X)))) xx)))) f((2)dz) 1/3
[([lFF,, forts Khy 182)
S ( ([(Kn -2)) 2))2dx)) f((z)dz)
$
(26)
have that 12 op (VFKAN &or (log(Fn) E + +)) Thus,
Therefore, by Lemma
1 F Op O, (log(Tk) 2+2 + +)) - ( 2+2
(n + 7) - (m 2,
(27)
Step 3:
Finding an upper bound. Now, we proceeded get an upper bound of 24 This is, an upper bound of the following expression,
Qa(m)) Qu(m + (m)) + Q' (nk + 7
(28)
24

---

Observe that, this expression can be written as,
(m())) Qu + + - (m()) Q" (nk +
me no
Σ
[| -Fss.mlAAilll Ilie +
[=
nx++
Σ
|| S(omm Khallis
Σ
|| [(neen Knallie
[m
1=
So that,
(xk)) - -Qa(n + - Q2 + (T: + U1 + U2.
where,
+7
+-
U1 =
Σ
|| - S(exina * Khg llin
Fe - F(sx Pesk.fe).ril and,
t=
I=
me+F
U2
Σ
|1 - Fei I12.2
|Faaa - f(mnee)) Khv
+=
t=
Now, analyze each of the terms above. For U1, observe that
The +F
|1 f(smm)) Kaallis - Σ
FL - -( ll2.2
(=
1=
[1 f(mmm)) Khulles - Σ
Fi - Fisiia
(= +1
1=
t++
|| Rooemel.hell...
Σ
[| P(on.ne).hall 1122
(= +1
1=
=13 Ta:
where,
Mi+
m
13
Σ
Σ
|e - F.oo.mml.h and,
t=+++1
t=
m=+
14 =
Σ
||
Σ
[F - Plon.til. 11
t=
tm
To analyze 13, rewrite it as follow,
m++F
13
Σ
[1
Khsa F(se.mu)hhe 12, -
Σ
(Rormme) Kiss - -F(ee.me)her Fe, - Soamm * Khal
(= +1
1=
=1a,1 Ta,2:
where,
Tie DF+7
13.1 = Σ
II
Krs Foononal and,
t=
DF
13.2 = -2 Σ
(Frcmml * Krs - F(su.mul.bess Fe,
t=
25

---

Now, we will get an upper bound for each of the terms above. The term 13,1
Op
( los(
which is followed by the use of Remark Even more, by Assumption 4 get
Ja,1 Op (p #+2
(29)
For the term 13,,, by Cauchy Schwartz inequality and triangle inequality, ([(emme) Kr ms Fi, [(exmma] * Kiaa L2
S (ex Khig me 11 Pahz m(( Khagllaa
< [esemm Khn Fise.mel.e 1z (11Frsh fmattee Koallia + I1Imm +1,eul Knz Sleut * Khalltz)
for any & (me, The F]. By the Remark [ we have that
(suoo Krza- Pienmmllaalles Op VT
(1T/
log(T)
Kig
and using basic properties of integrals ml * Kitaa
S1ou Krtallaa = O1Rx). Therefore,
I,,2 - Oz (
VTV
Tog(T)
(O(FF))
+
Σ 1=
|1 - ((owhoo)) * Kholles)
Rik
Now, we need to get bound of the magnitude of
Σ
[| - SmutLee * Khallea
in order to get an upper for la.2. This is done similarly to Iz. We consider the random variable
Y (Fi,1a f(nnee)) Khas Fu f(meou Kno) F((l -mun (Khallra)
E(I - -mu Khzzliia)
In order to use Lemma we observe that since [|Fe,b2 - Sm * KKnzlle2 20,
E(11 Pe,rr 8m Kbn 1122)
E(( S E[]] SmiI 11 Krn 11,2)
Therefore, using Lemma and that [B([[Fr,12 Sanl1 * KKaalll)
=0xxg
2"
by 26 we get that
E+2 F1 Sm+tiee Kna llEs = Op(V (KK (log(Fis)
1)) Op
(=
Thus, by Assumption and above,
log(T)
(O(Fk))+ +0, 7Kk (log(Fk) E+2 1)) +1) +,
3
Op E+2 22)).
Ta. S Op
Kig
(30)
Consequently, 13 has been bounded, and we only need to over the term 4, to finalize the analysis for U1. To analyze L1, we observe that
+7
1 =
Ferr - F(su.mlhhall2.
Σ 1=
+=
(Finey Ft 2(F For + mFloem hoaaaa La)
+=
(Fem Fem.) + 2(Ft,t,, F Flonnn))i - (F(on F Me
t +1
=I4,1 + I4,2 + I4.3
26

---

where,
1,1
Σ
(Fihar L2 ;
Le
t=
1,2
Σ
Fooe..e)hiie 2(F1, F(snma)) and, 2(F
t=
1,3
(e Re b2 (P(en. Me
tm
Now, we explore each of the terms 4,,, 1. and I,, First, I,, can be bounded as follows, we add and subtract (Fi,e, F to get
nu +
Σ
(F Fe,h2)2 - (FLM, Fumi
+1
Σ
(Pe Penz) - (Fensr Pemies + (Perer Penales - (Fats Le
1=
+C+F
(Pe,ha Petty Pe + (Pehis Ferra Pe
1m
which, by Hölder's inequality, is bounded by
T オウ
Flaalles 11Fe 112211 Fimille2 FOp(
log 3r4e (T)))
e 2
P
+1
Kk
)²). for any see Remark for more detail. Similarly, for 1,2, we have that adding and subtracting 2(Fr, h2
since |iit,
Fin
0 0( x+
Op (p(
((
Ing(T)
2
2r
log(T)
++
+
2(F -2P F(enmme).halt
- 2(Fe,
Σ
2(Fe,)) F(e. - 2(Fe, F(eemeehhal bo + 2(Pe, F(eeme).hal - 2(Fe, Foeeme)) ba
t=
Σ
2(FM, - Fenry F Fls + F Fisumeelhhhlee
1=
and by Hölder's inequality and Remark it is bounded by
[| Fr.nallra Feoummll. + |l (ee.mme) - Flee
+1
(( ((108(T) mpक
T Tht log(T) (Tg(T)) &
TIfTo log(T))) ))
=70p
+ x²
P +
KA
27

---

Finally, for 41,,, we notice that, adding and subtracting (F(s Rel.bi mogaia) it is written as,
nu+7 ++
[ (eu Pouumslibels (F(o Po
(F (elumme P(oumelibodls - (Flon Plox.nddddil
+ (e Plonmelhheles (Flen Flanmelhaale
(F(summ)iia Fis Fs + (F(sn F(auimelhh F(ss,fe),h))) ba
1=
which, by Hölder's inequality and Remark is bounded by
++
|| (sa
F(su.m)).nalle lak + 11F Bel llla (o - F(sn.ful.Milll
-70p
log(T) (
2rip
To+p log(T) $ )²)
A
A
Kkk
Then, by above and Assumption
we conclude
E+2 and (3T) we find that Un has the following upper bound,
Op( (FKg
(31)
From
29
30
|1 fisummll 11
Σ
||
F(se m.). 112, = (F(k E+2
(32)
t=
Now, making an analogous analysis, we have that U2 upper bounded by,
mo+F
|i F(i..eulMi 112.
Σ
Simeeul * Krz 112. op 2+2).
(33)
1=
In fact, we observe that
+F Σ
F(in.ex))h 117
| S(mmea) Knall?
+1
t=
|1 F(mnoe))
| f(mama) Knallfa
(=
t=
m=+
[| Finneeel 112
Σ
[|Fe, P(m..eu), 1
(=
t=
=Ig +
where,
m-+F
+F
Is =
Σ
Σ
|| - S(nuee * King 1ion and,
tmn +1
tmm
Th +F
16
Σ
FL - F
[F - F(mm
1=
1=
Then, 1ş is bounded follows
Tie +
15
Σ
[lf(n.ee) Kh2 Fo 112, +
Σ
(fim.cal Khs Fimmsss) Fi, - flance) * Khil
(=
t= +1
28

---

where,
I5,1
(n Khza F(m and,
t-F
15,2 = 2
(mn * Khz - Fumuieulihg Fe,ho fon Krnd
(=
The term 15.1 = =0p (++ log(T)
using Remark
Even more, by Assumption we get
I.1 = Op( +
(34)
For the term 1s,2, by Cauchy Schwartz inequality,
(f(nee Kba F oml Fi,ko Sun Kholls
$ [lfoonee)) Kha Flmmeeii, llza 1i floncol Khallez
for any E (me, Th + +.]. By Remark |
we have that [Ifeoned * Krz F Fimnenllhgle
Op T
log(T) (
Therefore,
(T)
Tik +
S,2 S Op
Σ
PL,ha Sinetl.ww) Knalles)
VT
Kk
+=
Now, similarly to the bound for 12, we consider the random variable
Y; (Fe,2 S(meeel Kw, Pan22 -((nee)) Kna) - E(ll Smn (Knallra)
E(11 Sme &Khh 1122))
In order to use Lemma we observe
E(1 Fe,to 8mn-1 Ktoz 112 E[] Fiiio fmnms Kbg l1,2)
E((Y,1 =
1.
so that, by Lemma 3
Is,2 S 5Op
1
(T)
(oplvingt (log(Fic ++2 ) + ++11))+ppppe Op(nx))) =p((fit +22). (35)
VT
#
To analyze Ls, we observe that
+-
16 =
Pe Fim.eellhh2l2 -
Σ t=
|i - For 112
#1
+F
(Pe Fenalis - 2(Fe, F.LFe..n,FFimeellla Fim.eel.halte + (Fimeeuldhs E Fonneeelhalt
1=
m++
Σ
- (Fehis FEmI) + 2(F F( (F(B F F(Bnee))aatt
1=
=16.1 + 1 + 16,3.
where,
#F
16,1
(F Fi - Fehaa
t=
16.2
2(Fr, Feneealhh - 2(FF Fon.eeal Lz
t=
16.3
(Fin.ewllhhr Fon.eshhbats - (F(n,ea)hh Eo
+=
29

---

Then we bound each of these terms. First, we rewrite 16,1,
n-+
F2 (Pe,e1 Pa
m
Σ
(a F2 (Fa,h Fanide + (Fihes Fiha (Fa Frnhe 12
t
Σ
(Fi F (FEM Fi, Fi
1=
which, by Hölder's inequality, is bounded by
Pi,ha 1112211/aa Pemilles + 1l Pl,
Pr,po
log(T)
T PT
log (T)
=50p
Kk
2r
KD
since Ft,x Fi 1i = 0 |
Op(
(og(T)
2F+F
T Tarto Tog(T) 1), for any see
p
Remark 2 for more detail. Similarly, for 1a.2 we have,
0k+
2(F F(nees)))) - 2(F Fx Lg
4++
Σ
2(F F( 2(F Fi + 2(F F((n.e)) - 2(Fa, F(m b2
t=
Σ
2(F Fibhy hu + 2(Fe, Piga.eul,h Fe
(=
and by Hölder's inequality and Remark] 2 it is bounded by
Fi Fe.tzlleg ll Finneellhally + ||Fe, ll ((niee))k Fx) 1l
t= +1
((
(L0B(T))
2n
T T log(T)) $) $))
=70p
+
KK
Now for Io,3, write it
(Fimeeullhha F Fonneellhea - (F F. Fon
t=
(Fine.eul.ber Fonneee)hhat (Frin.euJ.hr eon + (Fenreelhs Pon.eshhaa (Finee Pon..es),ba t=net
Σ
(Fim..e))h F(i b(s + (F(rn.e))b Fmncoube Fi
(-+++
which, by Hölder's inequality and Remark is bounded by
Fiih.eul.bill lle + IIF llke (f - Fiie.eul. llzz
(=++1
log(T)
F
T TI log(T) $
=70
KA
30

---

By above and Assumption
we conclude
le = op (TRk p +3
(36)
From,
34
35
and
36 we get that U2 is bounded by
75
Fi, F F.
|i fime.cul Kx 1its op Op
++2
+=
t=
Therefore, from 32 and
(33
$ ++2 (mp) Qn(m + Q" + Q" + 0p (FKg
(37)
Step 4: Combination of all the steps above. Finally, combining (24 27 and 37 any
uniformly for
we have that
12
Op O'(inann & (log(TRi 2+2 + 1]) - op(FKk ++3 Sop(FiE +3)
TAE
2
which implies,
FRi
++2
(1)
(38)
and complete the proofs of a.l and b.
2+2
Limiting distributions. For any & fl..... K} due to the uniform tightness of FM
24 and
37 T
(n)
Σ
|| floumml * Knollee +
F1,122 fimenea KKallea
tm
t=
satisfies
( (T{)) (Q" (TT +7) Q" 2-0(((( (Tx)) 1g O.
Therefore, it is sufficient to find the limiting distributions of Q" m + F Q- (m)) when T
Non-vanishing regime. Observe that for F >> O, we have that when T - oc.
n-+F
Q' + F) (1))))
Σ
|| -Son.mel Koall
Σ
|| [(ne Kng
1=
1=t+1
n=+F
[[(emmme Khgg Simeeel Khalle
+1
Σ (frsc.m) Koz flya.oul Khizs Fac flanc Khal b2
t=
2 Σ 2
Phrmette Smite Krtzs (Jris - Smm) Kht) + Fil(smet - Son Kno 112..
1=1
When F < 0 and T - oc, we have that
Q' + (m)) = Σ
FL - fiomnl Knall? -
Σ
|| - f(mera) * Knzlll
1=
(=
[)fm))) Khs S(mnral Khall?..
1=
1
(Steemml # Khs - Simereul * Khoz Fe. - Semnee * Khal
0
Σ 2
Fhematt - Smate * Khees (frr - fmez) * Kine) Ls
+ + - fmm x Kra 12.
1=F+1
31

---

Therefore, using Slutsky's theorem and the Argmax (or Argmin) continuous mapping theorem (see 3.2.2 Theorem van der Vaart and Wellner, 1996) we conclude
(Th Th m ++
2 arg (F)
(39)
FEZ
Vanishing regime. Vanishing regime. Let m = -2- and we have that m - 20 as T - 00.
Observe that for F > 0, we have that
Q% 7he + (mm) -Q +
(Tk)
Σ
(I Kha - flmm.col * Khe l12.
t=
Te +-mm1
2
(F(onona Kina fimeenl Khgg Fe, Simeeal Kizg Lo
Following the Central Limit Theorem for a-mixing, see Lemma 4 we get
The -om------
(Slenmme) Khg S(mneex) Khgg Pe,na Jmneee Knzl 2g ni (k)B((F),
#+1
K%
where B(F) is standard Brownian motion and (kk) is the long variance given in (8). Therefore, it holds that when + 00
Q (Th + Fm)) Q% (mx) 2, Kik 00 (r) + Frck -- 2II(exmma) * Kigg
[(me.ea) * Khy 11.2
Similarly, for 7 < O, we have that when n +
Qi(n= nT + +rmm Q% (k)
2 Ki o (-) - TKKK" -- 11 Khz S(meew Khy ll
Then, using Slutsky's theorem and the Argmax (or Argmin) continuous mapping theorem (see 3.2.2 Theorem in van der Vaart & Wellner (996)), and the fact that, ([]lf( Khz f(gneer) Kn,, l12,) (kK)). we conclude that
Kk 7Tk
116
2
arg min 0x (F) + (7). rEz
which completes the proof of b.2.
32

---

Proof of Theorem 3
In this section, we present the proof of theorem Theorem 3
Proof of Theorem
First, letting h2 Cu and R = 0(
T fr Kk
we consider
0% (k))
1
Y:)
where, Y, =x== ( Fhgit fo Khrn (fon -mu-) Khva) 1.2
S
(40)
We will show that
(1 T2 (A) - -20
T + oc, and
(ii 82 (l) - (k)
_ +
in order conclude the result. For (i), a² b² (a b) b), write,
22 82 (k)
SES
iES,
R
1
Σ
Y,
Y,
Σ
Y + Y.)
R
VS
r=
SES,
iES,
R
4
R
Then, we bound each of the terms I and Iz. For I1, we observe that,
1 =
Σ Y, ya VS
Σ Y
-X-l.
VS
IES.
iES,
Then, adding and subtracting, R -1 (Fhii fo Khis (fr -mn-i Khaa
and
b2
( (Fhzit fi Khz Fris + fo Krn, (frr finee) Kha) 1z"
we get that,
Y, Y:
( Fnggi Sa * KNyg (Jun Imann Khz)
#
-1
1/ ( FFi fa * Kays (Ser fin-.) Krns) Ls
L2
(Phais - S1 * Khas (Syn - Smn.1) ) & Khzz)
( "Fmis - Se * Khrs (So - I .) & Khnz)
K
+
F
Fiuss fo * Khis (frp - Smann Krts)
(Fhis- - fo * Krns (frn - Brunr) Krs) bg
Lg
$
Fay. - fo # Kin - Frin + fi # Krns (frn fir ) Knn) siz
# 1
F - S1 & Kina Frnn Si Krys (Jin Smen) Knn) La
33

---

which can be written as,
k Err
R,- - fs KhnD , (fr - -fattl) )+ * Kros (frn -funt) Krns )
(Fha,i fi K(2)
Rkk (Fhi.
La
+
k "(Fhn - fi KKn)
#= Rig
1 - fi Kna); (frr - fin ,) * Kins - -fmm Smn ) # Kho 1/2
(2)
1 -1 (Fhatt - fi Kha), (fne -fmmz) Kinz (frn - finte) * Kns)) L22
+ K4
+ 2
# "Fhhi - fs* Kba) #
(Fhaii fi Kha), (frr - finti) * Khss L2 Kiz
#
Now, we bound the expression above. For this purpose, by triangle inequality, it is enough to bound each of the terms above. Then, use Hölder's inequality. First,
1(Fhar - fc K(n2) - Ri "Fhin - f: Khr): (fr - -mutta Krzz - (frr - fanr1) * Knss in
(Fha,i
Iik
#
Rik
#
1 Fha.i fa Khha
Fris + Kre 1l L. Lall(fm -m--) Kitzz (frn fin--) Krn Lx
+ --1 Rx
11 Op
1 # ((og((2)
28
Tatt log(T) $). wn
Then, using 54 we have that |Kg
+
and using Remark 2 it follows that
|| haz. J& Kiza Fha. + Sa * Kh 1le S11Fhait Fheillees I1ls
Khs St Khalles
1
(( (og
Tz (T) $1 KA
=Op
+
++
A
lik
and,
11 - -outz) * Khh - (Pān Sints ) lez
S A1lmm * Khna - fi * Krillea + -lf * KKh - fmim * Khalles
1
log(()) 2rFF
To log(T) tt.
=Op
#r +
KA
Kik
So that,
K
#
-1
(Frai - Si Knra)= - -Rt
(Frni -
- S1 KKn)) (Son -Smiz) * KKng (Sān - S (n) * Kns) 12 1
Kk
Op(
1
((108(T)) 3 +
Tafto log (og())) $ 00p
1
(( ((108(7)) * +
Tafro log(T))) #)
2- #
KA
KA
Kikk
Kis
((1g) (ogg(T)
THix log(T)) $
Op
2 +
AA
Kig
Now, in a similar way, we observe that
(Fhaid fs Kha), (fm - SmHLI ) Khvs - (frr - S * Krns
Kg
< 1 "FFh.s - Si Kho)lleall([n - (1) Kb2 - (fār Sinmn ) Kralles
-Kp( 2. - 2 j0 (1
log(T)
srtp
Ta log(T)) bo
Ki
I+I A
+
KA
where equality is followed by noticing that
Frggi Sa Khp Knglliz
S1(Fngail 1.2 + [1[s * Knallra
(41)
=0(k; $ ( O(1).
(42)
34

---

and then using Remark 2 and Assumption 1 Finally,
k
#
(Fhi,d
K (Fhin fo KhzDs (fr. fanti Khns) L2 fo* Kr,)
<|R
-1
Kij
[|i,F - K(n) - (Flrzd - fo Khaa Kha))leell(fin - -fm-)) * Krn 1l :r-
(
TIrto log(?) $ 00
log(T) 2
T T log( (T) Aik
=0p p
2-1
+
A
K
K
KE
where equality is followed by Remark
Assumption
1
and Minkowski's inequality. Therefore,
5
1
1
Σ
Y;
Yi
V8
SEB,
VS(
0p
((
(log(T) (1')
Tw log(2))
2-
#
A
KA
Ki
-0p(
1
(( (log(2)
2-47
T2r+p (1)
})00
$
((
(log(1)
xt4p
T2r40 log(1) 1 )²)
+
A
#
A
KA
R
Kk
(N,
-1
T
2012
2r (T)) + Op(
T
4-
log
(T))O,(
1 2r+0
logarie 2rie (T))
2
+
K
Ky
Kk
T
log
1Op(
T
R+1 log(T) (T) ,+ + (x O)
T
log (T) .ল ))
2-
+
RE
14
-
1 (Fhait S Khrz (srz - S Kns / L2
To bound the 1z term, add and subtract
Ki
get
Y + Y:
K
-1
(hhii - Se # Khzzs (Fur - I'ne * Kha)
+
(Fii - Si * Knns (fir. - Jūx .) * Kmn
1/2 (Fhz.s for Khzs (frn fmena) Khza)
#
( (Fh fc Krzz (frn fin-.) Khi)
K
Kkk
1
( Fhag.s - fo # Kizzi (fre - Smemi x Ketes)
{ (Fm - fs * Knn (frr - finmi) & Kitnn Lo
+ Rik
La
-
#
"FFigs fe King (Jun Imnn Khbs)
+ 18k
f -1 (Frgxi fe * Kings (Jin fūn- KKhs La
L12
+
+( R
-1 "FFry - Si Khn))
Eo 8--1 (Phard + Khz)s (Jrn - (1m11) * Kms) ro)- ia
Ki.
Then, as before, we bound each of the terms above using Hölder's inequality. We start with the term
( (Fhgis - fi Knz: (Fre - F * Kira D2
Ki
2r
<x #
|| Ji Kha Khalleall(nn - Smin) Knallea
k
#.
Op * )Kn = Op())
Sx
where the second inequality is followed by (41). Similarly.
(Fng. fi Krzz (Jrr - fün. Krs) is inl #: 1| Ji * Khallkalll(e - S * Knyllza
< k
2.
Op(Rk )))RE = Op(1)
<x
k
where the second inequality is followed by the 441p. Finally, the term
( (R
"(Fryi fi Kn.)) x/ 9. (Fhrit fo Krn)) ) (frr Sine ) Knn) L2
35

---

was previously bounded by,
1
log (T) (T)
T オウ
p log (T)
0000
# 3 ((
(ogg(T)
(T)
2
T log (T) 31
Op
2-
#
A
K
A
KA
Kik
Kig
Therefore,
Σ +Y
12 S VS
iES,
=V5( +0p
2- (
1
(108(T)
2-10
2"
+
T log((T) (T)
))
0p
1
(
(1))(T))
2- +
T log(T)) $) }sss)
KA
++
lik
lik
In consequences,
R R
R
(b) - (k)
Σ S,S
2
1
1
Σ xi) '
=
R
VS
iES
R
I
1222
(s0pp =s
1
(To
2r+p
TF log(T
2
A
KA
0p
1
log(()
(T)
T log
00,0
+ (
(T)
3-15
T log
$1
+
+
B+1
A
KA
A
KA
Kk
Kik
(T
TzrTe log(
+0p p
2-
#
A
RA
Ki
1
log(1)
Tertp log (T")
((
(Log(T))
T T log (T')
Op
00
+
A
KA
A
KA
Kk
Kik
(log(T)
Ta log(T)
+0p(k 10p
++1
A
AA
Kik
# ((
(log(T)
Trix log(T)
(2)
+0p(kx 10
A
AA
Kik
(( ((oBg())
T F
log (T)
))0,,
( ((og(T))
Tattoo log(T)
$
2 Op
+
+
mik
KA
# +
A
RA
lik
ki
In order to conclude (i), we notice that by Assumption implies,
and that S = O(T c
+
which
(A) - o2) =0p(
Now, we are going to see that (k) -20 (b) 0, T x.. To this end, we will show that the estimator is asymptotically unbiased, and its variance + as + oo. First, we notice that, by Hölder's 's inequality and Minkowsky's 's inequality,
Yı
=|
-1
(Prai - fi * Knos (Fm - SSn..) * Khaa ) I is Se Kon Mealllaal1e Smlia) Kngllia
- *1
k
# = 1.
k
36

---

Now, we analyze the Bias, We observe that,
(()2 (K))
1
ΣE
R
B
=
1
Σ
8+1 --1
B(
S
iES,
iES,
14-
and.
5h (k)
Σ E(Y,(+)).
10-00
so that, the bias has the following form,
2 - -()))(( = 2 Σ B(YY +
E B(YY))))
2 1=1
S=
Now, we show that each of the above terms vanishes as T > xo We have that, by condition 2 and covariance inequality
2 Σ B(YYY++) 58
_
0, as T 00
i=
1=$
where or is the mixing coefficient. Then,
Σ l 1=1
Mill at S C S
<
Σ
0
1=1
by condition 2 choice of S and Assumption 4 Therefore, conclude that the Bias vanishes as 2
- oc. analyze the Variance, we observe that, Y = 1/2 (iis, Y.)
Var (k)) EE( (k) E(03. (1:)))3 (y,)((,)) )
1
Σ Y,
R
n-1
R-1
Σ
coou(Y, Y(4))
1=-R+1
2
8CS
S
0, as, T + 00.
R
1=0
where, oy are the mixing coefficients of (Yr) which is bounded by the mixing coefficient on From here, conclude the result (ii).
37

---

G Large probability events
In this section, we deal with all the large probability events that occurred in the proof of Theorem Recall that, for any (s, e] _ (0, T}
t
fe (x) = V (e s)
fi(x)
X. fi(x);
1 (e s)
1=1+1
Proposition 1. For any x,
(Kalo- Xi) { Kn((0=2)dFF) CV
log T
P
max PEKST
$ T-P
hp
t=7+1
Proof. We have that the random variables 174= = Kr (x- x,) | satisfies
o (v((x-X))) co xx)
and,
Kn (1 x.) $ hv
Ck. Cx.
Moreover, let
V2 sup var((Kn - X1) + 2 Σ t>0
|c00( Ze, E,)D)
fot
We observe that,
1
1
x X2
x) ))3)
var(Kn(I - -xx) SE((
KC[
h8
h
/
1
x = )dF((z) )
<
K((
h2p
h
making μ 2-2 the last inequality is equal to
1 12p
KC( x - (dF((()) - hp / (((w)dF((())
$ hr CzCs-
Then, by proposition 2.5 on Fan & Yao 2008 ) |co0 Zz, Z1 + t)] S C C2. C2. On the other hand, coo (1, 21 +1) -IE(Z Z141) -(()))
< / / Kn(x ((((x-=2)KKnll 22))) + E(Z1)²
1199e I1 + E.
Since by assumption Assumption D we have that lloellero < oc, and
/
KC x )dF((2) = / K(u)f((x- hu)du = 0(1), hp
1
=(Z))= = Ex X1))
h
obtain that [cov 1,, Z ). Therefore, condition bound, inequality 2
Σ lcoo( Z1, Ze+1)1 < C and, using the mixing
2Ct
lcou( Zn, 71 1) <D
h2p
20 1/v
e
h
ha DhP
hppp
38

---

where the last inequity is followed by the fact e < for > -1. In consequence.
V sup
vur((Ks Xx) 2
Icou (z, 23)1))
130
jot
=c -
-
h8
Then, by Bernstein inequality for mixing dependence, see Merlevède et al. 2009) for more details, letting
A= == (v
klog(T)
log(T) hppp
log(T) log (og((k)
+ V
hp
he
get that,
k (nnl0 (x(x xx) - / Knlo Kalo==))F((()) >1) ST-P-3
P
in consequence,
P((
Σ (x -x)) - / (c((==2)dF((3)) (=++1
F+
> A S STPS
Since khP 2 (Tg((T) if p. and log2(k) = O(k),
Cp(( (v (lo((T)
log(T) h2
log(T) log 2 (As)
AP
Vk
V/:
log(T)
log(T))
log(2") log (k) 2k))(k) kh
-Cp
khhpp
Tog(T)
log(T) h0
<Cp
h
log(T)
SG1 /
hP
follows that,
F+1 Σ F=
(Kulo- Xi) / (o)((=))dF(())) > O.
log(T) hr
S TT-333
Proposition 2. Define the events
log
20 2/1VP hr + 20 202VTH
A
max
sup Pi (x) Je -() 2 20
hp
DER*
and,
A2 =
max n ZERP
sup
F+R Σ (xo) 20
log T
Civp h
+ C2VTH
hp
tm7+1
Then
P(Ai) < 2RPT-2
(43)
P P(A2) RPT-2
(44)
where R. is a positive constant.
39

---

Proof. First, we notice that
7
1
Σ Kn (0- s1(00)
max sup PEKET-PDERY Vk
1=++1
(Ka(x- Malx--00 X1) xx) - (=)((==>)ddttt =)dFi(z))
max sup P SERY
1=+11
max sup PEKET-PIERT ERR
Σ 1=11
= + 12
Now we will bound each of the terms 11, 12. For I1, we consider
A {II...., ".....,"(((. (RVT///heR))
with (B/T Reo(r; vizh J D, where D is the support of K and Ree(n; vhph
boxes centered I, of size /hch and R the size of the boxe containing D. Then by Proposi- VT
tion T for any Ti:
1
F+
(Ka(2,- Xx) - - ()((==)))F(((2)) 2
log hp < T-
max
CV
P-
PSKST
VK
1=
(45)
by an union bound argument,
P max sup PEKET-izEe ZEA Vk t=
i+k
Σ
(Ka(0 (x)) x1)- - Kall Kalo-=)aaA(()) 2 cv
llog T
hr < P- |//.
(46)
Let 11,1- {maxpekgt=2 Su A
vk Ei= (Kall X.) -
- ( For any
E R", there exist Ig E A such that
1 F+*
Kn(a - xi) xi)- Kx(x--x))
(Kalo==)aRiA
max P
Vb
F+1
(Kn(1 - -x,) - -Kultss -x((===)dd((())) 2)dff(z))
max
Vk t=
F+l
(Gn(x-Xp - - -Kn(ss - -x.)
max
VK
t=
F+*
( =)dr((z) - - Kalt (c((==2)dF(())
max
P
t=7+1
1 741 (KC((13- (x,1
xx) 1 /Kalos 2)dF((z))
sup p SEA
max
1=F+1
F+k
(K((x-x)) - -x))
max P
Vi
(=F+1
F+R
(fKaler- - 2)dF((3)- - [Kn(o=
max
Vi (=P+1
- + 11.2 I1.3
40

---

The term 1 is bounded as followed.
F+
( -K)le=
-x, - -x:))
max P
1=++1
7+ Kn(( -x) Kx; X.)
max PERST-P
1m741
F+k
1 Dz
PERST- Vk
max
hp+1
1m741
7+k
VIPhVP
VP Vhp
max
RERST
V
VTh++1
For the term 4,3, since the random variables {Ka(x - x,))T=1 have bounded expected value for any x RD
F+t (SKaCe- -2) - (Ka(t=
max
PEKST
1=F+1
F+tk
2CVV
max
vhv
t=F+1
Thus,
1 T+L
(Ka(1s X.) - / Ka(ls=2))) 2)dr(())
Σ
max sup
PskST - #ERR
Vk t=
S111 + C2 vp
vh
From here,
P(1 > Cip / logT
C2vP Vho
logT C2vP
P(1>C1//
+
SP [1,1 + 11, + 113 CiV
(47)
hp
hp
Vho
SP <P(Tia) [1.1) S 3 A| ( = R
//hhh/
(48)
(49)
Finally, we analyze the term 1z. By the adaptive assumption, the following is satisfied,
r4
(f (fKa(=-2)dFF(C 2))Fs()) - (f)())))))))
Σ
max
sup
P DER'
1=P+1
r-
[ [Kule=>)des - fc(z))
Σ
max
sup ZERP
PERET -7 VT
1=P+1
r
max PERET-
1
Σ
O2h
Vk
1=7+1
VCT
We conclude the bound for event A2. We conclude the bound for event Ag. Next, to derive the bound for event A1, by definition of Fare and fc we have that
F. F2 (2)
e e-t
(an(t)-fn((
1
(e s)(1 8)
1=s+1
(z) - (La(t)))
+
(e s)(e - 1)
1=t+1
41

---

Then, we observe that,
1 -
t t-8 s
(c s)(t - 8) 1
i S t, and
$
Ve-z
if s.
(e - s) - 1)
Therefore,
X 1=s+pt1
E-p max
Flaee (x) - (x)) S
max
1
Σ
- -
1=++1
(Li(()) ((a(o))) =X1 + X2.
max
t= -+p
Finally, letting A = 201 V
Ing T hP
+
202
+ +2"2VTh", we get that
v/r
P(X sP(XI X2 2
SP(Xi ス \ +P(X2 2 )
<2RTT-3
where the last inequality follows from above. This concludes the bound for A
Remark 1. On the events (Ai)e and, (A2)s, by Assumption 7 we have that
max t=+pp+
[[ Fe (z)lla <
max tstpp+
Öx sup ZERP
( - (0)
logT hp
20xCIVP
<2OxOV
20xCNVTH'
hp
where Cx is the volume of the set X. Moreover, using inequality
(47 we have that
1 F+*
Σ
(Ka(-- xx) - ( L1
max P F Vk
(=F+1
7+* (Knl (xi) - -Knn(t- 2))F(())
<Cx
max sup SENST-FZER?
t=7+1
-0x (v (1o
1log hr
42

---

a-mixing condition
process (Xe, t 2) is said to a-mixing if
ag
supa(o((X,) S
k)) k4000
The strong mixing, or a-mixing coefficient between two a-fields A and is defined as
a)A,B) =
sup AEA,BEB
I) P(())(B))
Suppose X and Y are two random variables. Then for positive numbers p- 1 + 9 + +r-1 = =1,1
holds that
Cov(X, S 41|xX|2,,, (al
Let { 13 ) 10
s==00 be stationary time series vectors. Denote the alpha mixing coefficients of k to be
a(k) =a ... Z0-1 2.).0f 21t 2140-1.....) }).
Note that the definition is independent of t.
H.1 Maximal Inequality
The unstationary version of the following lemma is in Lemma B.5. of Kirch (2006).
{
Lemma 1. Suppose
is stationary alpha-mixing time series with mixing coefficient akk)
and that E yi O. Suppose that there exists 6, A > such that
2+6+A
B y1
S D1
and
(k + 1) A122446))) < Dz.
R=U
Then
d
248
E
max
< Dr (2+51/2
where D only depends on 6 and the joint distribution of {(: 00
Proof. This Lemma B.8. of Kirch 2006).
Lemma 2. Suppose that there exists 6, A O such that
2+8+A
y/
S D1
and
(1: + (xk)) (1(2))((+0))) < Dz.
k=
Then holds that for any 0,0 and 0.
Σ M/
IP
max KE (d.a)
2x) < Cr
Cr--2-1
Vi:
where C. is some constant.
43

---

Proof. Let
S
V1
k=
Then Lemma 1
implies that
58
S SQidt12
La44
Therefore it holds that
P
S
2 2
=P
S 2+8
2 x 22
$ C1x -26
Observe that
S
Σ i== Vi
Σ Li=1 y1
E Vi Vk/v
max
max k pd
max
vd
k= l.....
Vd
Vd
ke (dd.d
Therefore
E/= x\VVV
P
max ke d
SP
2Φ) $ -12-2-6
Vk.
which gives
Σ yi
max kE(vd,d
2 2)
< Cza C2x-226 -2-
/
Lemma 3. Let v > O be given. Under the same assumptions as in Lemma 1 for any 0 < a < 1
holds that
PPO
yı
VFllog(rv) 1) for all r 2 1/v) 2 1 a²
where some absolute constant.
Proof. Let Zt #andd Ts 22/vv20H0 /v
By Lemma 3, for all x 2 1,
Σi=
2x) <Cp 2-- s C1
-2-5
sup reT,
Vr
Therefore by a union bound, for any <<a <a<k,
Eiath
VC (s+1)
a²
P 3s Et Zt
a²x²/0
sup rET,
Vr
a
(s 1)2
For any r 22/vv, 2P//u,et11// 20t2/ /v] < log(rv)/ log(2), and therefore
Σ Di
VC1 log(rv)
P (333s 3s E z+
+1]) < 7² 1og(2)
sup EET
/r
a
Equation (2) directly gives
Σ Vi
sup rET.
2
(log(re) + +1}) s a²
r
44

---

H.2 Central Limit theorem
Below is the central limit theorem for a-mixing random variable. We refer to Doukhan 1994) for more details.
Lemma 4. Let { % be centred o-mixing stationary time series. Suppose for the mixing coefficients
and moments, for some 8 >> O it holds
Σ
8/(2+)
E E
2+5 < 00)
ak
<0,
Denote Sn = Ei= Ze and of E
Then
S(ntl
+ W(t),
where convergence is in Skorohod topology and W (1) is the standard Brownian motion on [0, 1].
45

---

I Additional Technical Results
Lemma 5. Let I be defined in Definition 2 and suppose Assumption 7 holds. Then for each
change point Th there exists a seeded interval Lt = (8k, Ck) such that
Ig contains exactly one change point qui 17k A; and
b. min(mx- Sk, Ek
max [TK Sk.
A:
Proof. These are the desired properties of seeded intervals by construction. The proof is the same as theorem 3 of Kovács et al. 2020) and is provided here for completeness.
Let & E {1,..., R), where ft is from definition 2 By construction of seeded intervals, we have that in Jk. we can find an interval (l(i 1)72-* [(i 1)72 + + 1t],+ for some i E1,... 1..... 2 - 1, such that
4
min [(i 1)72 J, [(i 1)72 T2 -2- m/3}
(50)
and
max - L(i 1)T2 (i 1)72 T2 1k S 4.
(51)
where 14 = -2-2++++ +++ is the size of each interval in Jk. By the choice of R, there exist k such that 4. 1o A, from where the claim is followed.
Lemma 6. Let {X:} be random grid points sampled from common density function Ja IR" R,
satisfying Assumption a and -b. Under Assumption (2). the density estimator of the sampling distribution
T
f((z) = = Σ T 1=1
Kn - -x), DER.
satisfies,
|| fillez (
log (log
2c
(52)
T
The verification of these bounds can be found in many places in the literature. See for example Yu (1993) and Tsybakov (2009). Remark Even more, by Assumption 7
II falll,, S Cxll fellezz = o(( log(T)
(
Frip
(53)
T
with high probability, Therefore, given that
The 11 2
Σ
-i=m +1 fillez
Met
(1k 17
(Te
() -0
(54)
1-7h)
and 5 by triangle inequality,
and Lemma 5 have that
K R Op log(T) From here, and Assumption
Te log(T))
2
KA
h = O11+)) and h22 = O(Rt), we conclude that
|| - Fe, =0 IN
In fact,
|| - Fe.nall.
Xa
T. -Xi 1)
hpp KC( hn
KC(
By
h
h12
KX( pa-xy
X,
x)))²
K.( x
- X
KCl x xxx-xx
xx)+
KC( x Xx) x)) dx.
2
+
h1
h
h1
heg
h
h2 h2
Ry
46

---

Now, we analyze the two two following terms,
1
X1
2
1
2
-
Xx.
1
X1
11
K(
)
RP
h
h1
h4
h1
122
h2
and
12
1
)
x
K((
Nx-
1
K((
x
Xt
)x..
182
h22
1
h1
h12
h12
For 1,, letting u XxX,,
we have that
h1
f
(
x
x)) dx {
X1
1 (x(u))
du
Ry
JRP hi
and, letting Z-Xe, have that
Je
Xx
1
x
X dr
1
K(u (((u)do.
h2
h
h1
hly
h2
hy
Therefore, by Assumption and the Mean Value Theorem,
11
1 10
/
K(v) (K(()) - xC(v
h2 h1
du 50
1
h2
K(w)llolldo
l
h11
R2
R2
1/|
h22
0
hpp
1k K
Rd
=0
R (11-1
0
Similarly, we have,
1 Jay
1
K((
x -
xx,)²
K(
x
Xx
K(
Xx,
h2
h12
h1
h2
hz
1 [ K(v) (K(()) k(v 111 hz
h1
/ Jgc
)
R k
du S C
1
K(v)lluldde
h2
h2
h2
L1 Multivariate change point detection lemmas
We present some technical results corresponding to the generalization of the univariate CUSUM to the Multivariate case. For more details, we refer the interested readers to Padilla et 2202 and Wang et al. (2020).
Let (T,)T= C IR process with unknown densities {fe)T Assumption 5. We assume there exist { C =2,,.., T} with 1 = < <m < < 14 S <
MK+1 7 such that
fi + fit1 if and only if E {m.... nk )
(55)
Assume
min (Tk 17--1) 2 A 0. k K+.
< [| S 1lo ky for all k I.....K K.
In the rest of this section, the notation
7. (x) 1 (e st 8)
E Σ f5(x) - (ee-)) s) - 1) j=t+1
t (
Σ fz())
j=8+1
for all 0 05 <t<ttess T and RR
Lemma 7. [f e] contain two and only two change points тhe and 17 then
| lle, S -o--+illr+r frlle, + Vn. - f--1l/1g
sup Si0
47

---

Proof. This Lemma 15 in Wang et al. (020))
Consider the sequence (s) _
{ s=st1 be such that
{
fm
if 1 < t < T7 The St e.
fcr
For any 2 17..
Fe 9t
7k
e
fi
Σ
Sm
Σ
fi)
(e st
1=0t1
i=+t
i= m
t
fi
Σ i=++1
fi)
V
(e 8)(e
1=1+1
e
(Th T s)
(fm fme-.).
V (e s)(( - 8)
So for 2 7k: 11,e Te 1lz
Shik.
Since supasise 1112
max {11Fx Logz fl [1zz }. and that
max { [11/ 11 De4s [1aa)
S sup
1106
11 + VII 8K%
SSrSe
S S 7 VTr SKE
where the last inequality follows form the fact that gr. has only one change point in [s,e].
Lemma 8, Suppose
S CRA, where CR > 0 is an absolute constant, and that
a Σ $ S me S ... S metg I e S ne+++1, 920 0
Denote
f -m (2) k S k
max max
sup ZERP
Then for any k --1 1spp S k + q. it holds that
f((a) fm (x) I Cn
sup RER
Proof. This is Lemma 18 in Wang et al. 2020).. Since S Ss CRA, the interval [s, el contains at most Cn + change points. Observe that
fc
far Lx
1
1/6
fm- - fn))
+
Σ
(Fre - fm/ + +
Σ
(Fmen - Smm) Iike
Li
8
i=8
b= +1
i=
1
Ip
k) wiix
TK-1
Σ
1p- k
1 + +
p p 9
|n: Kmmax
C
1=8
1=
i=
m=+
1
CR + 1) Nin
i=s
where
P1 P2 & Cn + for any Up Up E [8, e) is used in the last inequality.
Lemma 9, Let (s, c) C (0,n) contains two or more change points such that
Mk-1 I 8 S mi S ... S M I e S M]+g+1, q a21
If Th - A. e 0, then
Fe.e
ja.e
+ 2 /
Lo
1.0
48

---

Proof. This Lemma 20 in Wang et al. 2020).
Consider the sequence ( {
such that
c=s+1
fnc-1 ftr
8 8 ++15 ++ists 7
gt
7/K + +155se
For any 1 2 T/. it holds that
11 - 1rm =
llv
(e 8) -
?11 - 8) (Jm.11 Smc )1 1/
S VTIk 8KCk
(e st 8)
Thus,
?lk - 8) e -7]++1
11200 1 11 Lx + VTk - Stik 1
11 11 + / SKi
[+++1 8 T)k
C A
1oghe Ill. + VT skip /aall fare
11 + 2 Skik.
A
where the first inequality follows from the observation that the first change point of yr in (8, is at
1-++1-
Lemma 10. Under Assumption 5 for any interval (s, e) (0, T) satisfying
Th- S s me S ... ...S. Ti++ S S Th++++2> 92
Let
[x)
b E arg sup 1= ZERP
Then b {n.....uk) For any fixed E TRP, if Flo.e( > for some (8,e), e), then
Jt [( is either strictly monotonic or decreases and then increases within each of the interval (T,, Mb41 ...., e
Proof. We prove this by contradiction. Assume that b & fm...., Let 21 arg max
Fr. (I) (x)
#E3r
Due to the definition of b, we have
P6 21 /-
b bE arg max
1=
It is easy to see that the collection of change points ((( l 1i is subset of the change points of
(f) Then, from Lemma 2.2 in Venkatraman (992) that
Ma
21
max jE(k....ktg!
Fla,el
21
I
max
sup
ft
Flae
(x)
t=st
DERP
which is contradiction.
Recall that in Algorithm 1, when searching for change points in the interval (s, e), we actually restrict to values E (s A, -p). 0). We now show that for intervals satisfying condition SE from Lemma
1, taking the maximum of the CUSUM statistic over (s + p. is equivalent to searching on (8, e). when there are change points in (s ppe-
Lemma 11. Let 20 E RP, (s,c) C (O,T). Suppose that there exists a true change point di E (s, e) such that
min { 11k 8,
11k a
(56)
and
Flaiel n
(20) 2 (c1/2)
FA
(57)
Ve
49

---

where > 0 is sufficiently small constant. In addition, assume that
F(s.c)
2
(0))
S cz 5) -e>
5)-3/2k,
(58)
where 02 > 0 is sufficiently small constant.
Then for any d E (8, e) satisfying
d
M)
- c₁
A/23
(59)
holds that
Fls.el
20 1 F/
(0)) > c Tk
A Pizz
(00)
(e
s)=
(60)
where is sufficienttl small constant, depending all the other absolute constants.
Proof. Without loss of generality, we assume that d 2 7/8 and F
20)
2 0. Following the arguments
in Lemma 2.6 in Venkatraman (1992). it suffices to consider two cases: (i) 77--1 > e and (ii) T S
Case (i). Note that
F(s.el
20) = V
m7
T -5
[nm (20) (20) -f 2 (20)}
and
[/ 20
Mp - $)
- d { nna (20) -Sness (00) }.
(e 8) ( - 8)
Therefore, follows from (56) that
(e - d) 9/1 (8)
Fle.el (20) - (00) = =(1-
sl
Floie) (00) 2 c d - m (e-s)-2fie 2f (20):
(d T7K
(61)
The inequality follows from the following arguments. Let Then
7e - 8, 7K and wr d 1k-
(e d)
17k -
c d - me (e - s)²
(d s)
(e- 1166
(v w)u
Aw
=1
(u + w)v
(u + 0)2
w(u + v)
Aw
Vuu w))(v (v w)u v(u w)v) (n + r)
The numerator of the above equals
w(u + v)3 cdu(u + wiy cdw uulu + w - ww)
2 2c, Awf (u (u+ v)2 (u+wwv c/uo(u + w)(w w)
201
201
> 2c, Awa (1 (1-c/ c/(221) (u + u2² 2 20---211/2 2-1/2c/c2ue
00
as long
V201
c
4+ 1/ (v2c))
Case (ii). Let g 0/ A/16. We can write
Fla,el vik
Flaiel (20) (a + +90)
20 a \
Th
178
1
e - 9)
1Th +
50

---

where
1k
a =
Σ
S3 20
Σ
f3 (20)
j=0+1
1=0+1
0 a nT + (g--)) 5) ce 7] - 9)
ay(na
{
1
b
+
g
V (Tn - 5) (e n&-
mh + 9 9-3---0 -3) -) 7/ -9)
ave
and
b pls.e] (20)
no
fike
(20)
To ease notation, let d T)k -ls 9/2, Ni 11 and
N₂ ?k
We have
Ej
Flaiel (20)
Flane] 20 E1i 1 F2) + Est.
(62)
where
01 - -Ive-8 -8
Eu
/M N2 M)++ ((N) +1) (9 g+ N2 (2) (M)(((+))))))))))))))()) No No- -i) + v vN (g + N22
(i
NN2- NN1) (No- -N2-N1 -N1-1)
E21-
+ (1
9)a) (v )))++)
+ 9) Na)
(g+ + +V(N + 9)
+
and
N1 + N
bl
Fxs
g (N₂ 1) (g + N₂ 11)
Next, we notice that g 1 12 c It holds that
E 2 cu d - 7h
A [la.c]
20 (e s)=
(63)
where en > a sufficiently small constant depending on 61. As for Ezt, due to 55 we have
Ex 2 11/2.
(64)
As for Est. have
Eu 2 -ca,, - 1hk (e 3) -c3,2bld m A -e 0)3/2 fla.el (00 (20)m=1
(65)
2 -21/2 1h2
A
Flane]
z0 (e - 5)
(66)
where the second inequality follows from (5) and the third inequality follows from are sufficiently small constants, depending on all the other absolute constants. Combining (62).
58
C31,1, C31.2 >
633
(4) and 65). we have
Fla.el
20
20
c
d
17
Afls.e)
Ti
20 (e
s)-2
(67)
where e > 0 is sufficiently small constant. In view of 61 and
(67 the proof is complete.
Consider the following events
A((s,el.p.)) =
F (2) - (2) s $1)
max (Estptl.... ZERP
sup
r+N Σ
(Fe, 11) }
B(r,p,7)
max
sup
N=p....ff -T ZEER
VN
U
Σ
sup (F(, f((e)) s7]
max N N=p
VN
-+p-N+11
SER
51

---

Lemma 12. Suppose Assumption 5 holds. Let [s,e] be an subinterval of [1. T with e CRA, and contain at least one change point Thr with min nr s,C nr 2 cT for some constant
Let i max [Kip min s,C hp} 2 c.) Let
b arg max P. P
||
For some 01 > 0, A > and > 0, suppose that the following events hold
A((s, ((
(68)
Bss, p,) U p()
U
B(n
(69)
and that
max P
P1 112 =11
2 ca S. VT
(70)
If there exists a sufficiently small 02 > 0 such that
S 02 VT and that
p S CTT,
(71)
then there exists a change point Tis E (s,e) e) such that
min(e 7e, T
$} cgT and |m bl C3 max( xxp² ,p.
where C3 is some sufficiently small constant independent Tf
Proof. Let 21 arg maxxeR> [ (=) Without loss of generality, assume that Pl (2))
(2)
and that 7( [1 as a function of t is locally decreasing at b. Observe that there has to be a change point 77 E (s, or otherwise F(s.el
21 > O implies that [! [1 is decreasing, as
consequence of Lemma
10 Thus, there exists a change point 17 E (s, satisfying that
sup =E
T1 el (=) 2
Fla.el
21
N sup
(((.)) - 7 2 CK A
(72)
where the second inequality follows from Lemma 10 the third because of the good event A, and
fourth inequalities by (70) and Assumption and 0 is an absolute constant. Observe that (8, e)
has to contain at least one change point or otherwise supzeR fT
7[s.el
(=)
which contradicts (72)
Step 1. In this step, we are to show that
min s, mh min { A/16
(73)
Suppose that We is the only change point (8, e). Then 73 must hold or otherwise it follows from (14) that
C1 A A
sup
fa.e
S Kk
SERP
4
which contradicts
Suppose (s,e) contains at least change points. Then arguing by contradiction, if 7/
min 1, oi A/16, it must be the cast that 7)s is the left most change point in e). Therefore
sup =ERP
ss.e (z)
c /4 sup ZERP
Jse (z)
+ 2NL Th -
(74)
0 /4 max sup stpetse-A seR
(x)
+ -
(75)
< 0
max sup stpckcep SER"
(x)
+ -///47 + 22 Any
(76)
< sup =CRP
G3.e (z)
(77)
52

---

where the first inequality follows from Lemma 9 the second follows from the assumption of 77 the third from the definition of the event A and the last from (70) and Assumption
TT
The last display
contradicts (72), thus 73) must hold.
Step Let
20 E arg max (ite(2) (x)
SERP
It follows from Lemma 11 that there exits d E (Te. 7k cr14332)
such that
fsie 220)
fa
20 2 2%
(78)
We claim that b
(ne:)) d)
<(TE,
me. 7k + A/166. By contradiction, suppose that b 2 d. Then
f 20 S Pa
20
S
max sup Scice ZERP
c (z)
2 S
sup =ERP
F3 [x)
%
(79)
where the first inequality follows from Lemma 10 the second follows from 78 and the third follows from the definition of the event A. Note that 79 a contradiction to the bound (72). therefore we have b E
Me. Th + 43332).
Step Let
j arg max Fo (xG))) pa j=l....T
-Istt
(((()))...,(((())))))) R((
and
X 5 x X(s)
x x())) - x(e) )) R(c-s)
F 1
1
h
By the definition of b, it holds that
1 p Pp (pair)
12 (
Fe. Par
(ps))) ||
F
)
where the operator (+) is defined in Lemma 21 in Wang et al. (2020). For the sake contradiction. throughout the rest of this argument suppose that, for some sufficiently large constant C33 > 0 to be specified,
Vik + Ca <
(80)
We will show that this leads to the bound
|| | p. )Fa:))]²
|ose | pi (foxt)
(81)
which is contradiction. If we can show that
2 par_ pame-feee fee,, pr (msse) ( pa.e (pac)) <
|| - n pre(Fat))) (82)
we first note that min {e-nk Th - 2 min { 1, {1,2]///16
then BTp holds. To derive (82) from 80
and that b- - mk S c. A //2 implies that
min(c- b, b - s) min (1.6)) c///3 min {1,tt A/32
(83)
As for the right-hand side of 12 we have
I | p/ (fase) )1² pr. Dar )1²
F (x ()+))) - ( (x (x()))))))
2
(84)
2 j (x(5))) Jo. (x(x*)
je. x
xx+*))
(85)
53

---

On the event AnB B, we are to use Lemma
11
Note that 57
holds due to the fact that here we have
f x (x()))
2 ps.e ( (x(i+)) F x (x((5))
2 c 2 (a)) (2))/K1 /
(86)
where the first inequality follows from the fact that Th is a true change point, the second inequality holds due to the event A. the third inequality follows from (70), and the final inequality follows from (II). Towards this end, it follows from Lemma that
156
[ (x(+)))
c/b Mc
S
(x (x())))
1(e (e- (87)
Combining
184 866
and (87p. we have
fare Po (fate)
far - - (pxc) [|
2
coi A - -))-2/0
b- m//
(88)
4
The left-hand side of (82) can be decomposed as follows.
2 Fsre - p , pi
psy (f(-e)
(89)
-2 F.0 - - p (
p (ma)) ) + 2 yai (/--00--0 -s (sar) -par (ppe(par)) (sarr)
(90)
8
=(1)+ +
Σ
pss pai)) ) (p(ce(2ae) p (sar²)
(91)
i= s+1
b=b-
=(I) + (II.1)+ + (II.2) + (IT.3).
(92)
As for the term (I), we have
(T) 272
(93)
As for the term (II.1), we have
-8
(II.1) = 2/ Th4 Tik
pan - f3.0
g ps.0
(fant),} }
(far))
1/k
1=1
i=1
In addition, it holds that
1
7/4
b - 1]
1 7/4
fi
(x
(x()))) + fm (x (x()
b 1=1
T7k
6
b Th Cn
max $0.000
b
where the inequality followed by Lemma 8 Combining with the good events,
(11.1) S 2/m
b T (CR +
(1t++1)xmm k 7
(94)
b
4
1 -1/2 17 (CR + ff)++xxx00 Ni00.00
2 min 11
(95)
2
As for the term (II.2), it holds that
(II.2) s 2
b mh
/ (222n +3) max
(96)
As for the term (1I.3), it holds that
(11.3) < 2
3 1/22/1 Tp (CR + +1)
1
max s0.c0
(97)
min
1,
2
Therefore, combining
94
96
97
88)
89
and
93 we have that
82 holds if
A 2 -
s)
7k
max { A 1/2y me Ne.
0
ns yrak
(98)
The second inequality holds due to Assumption 3 the third inequality holds due to 77) and the first inequality is a consequence of the third inequality and Assumption 3
54


Reviews:
[{"review_id": "X9e6x9DyA1", "reviewer": "Reviewer_XKms", "summary": "This paper studies non-parametric offline change-point detection with the assumption that the probability density functions are Holder continuous on some compact support, and the time series is $\\alpha$-mixing. The authors propose a two-stage algorithm, first roughly divide the time series into different segments, then refine the change-point locations. The asymptotic distribution of the estimated change-points are derived, and as a third step, the author also proposes an algorithm to compute a confidence interval for each change-point. Numerical experiments show the effectiveness of the proposed methods.", "questions": "1. In Assumption 2.a, is it for any $h>0$?\n2. In Assumption 2.b, the end of line 155, what is $v$ on the exponent? Do you mean $\\nu$?\n3. In Theorem 2.a the non-vanishing regime, do you need $f_{\\eta_k},f_{\\eta_{k+1}}$ to converge in distribution as well or they can be arbitrary as long as the jump size $\\kappa_k$ converges to a constant? How does the $\\alpha$-mixing works here? What is $F_{t,h_2}$ for $t<0$? Why the limiting distribution of $\\widetilde \\eta_k$ depends on $f_0,f_1,f_2$ when $\\eta_k$ is far from 0?\n4. In equation 7, should it be $\\kappa_k^{p/r+2}$ on the numerator or just $\\kappa_k^{p/r-2}$?", "limitations": "As stated in the submitted paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The non-parametric model used in this paper is Holder continuous class and is general. \n2. The asymptotic distribution of the estimated change-point location is derived for the vanishing and non-vanishing regime respectively.\n3. The time series can have temporal dependencies under mixing conditions.\n4. The writing and the structure of the paper is clear.", "weaknesses": "1. The authors claim their model is under mixing conditions but the theory parts' dependency on the $\\alpha$-mixing coefficient $c$ in Assumption 1.e is hardly discussed in the main text.\n\nMinor issue: \n1. The equation under line 126: should remove $\\arg\\min$. \n2. The equation under line 126: why is there $\\eta_k$? Isn't $\\eta_k$ unknown?\n3. Line 131: Why is $\\widetilde \\eta_k$ referred to as the `kernel density estimator'? From previous context it is the change-point estimator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive comments.  We reply to your comments below, with corresponding edits in the revision.\n\n**$\\alpha$-mixing coefficients**\n\nWe appreciate the reviewer's comments regarding the need for a more detailed discussion on $\\alpha$-mixing coefficient $c$. We will include the following discussion in our revised manuscript.\n\nThroughout this paper, we focus on multivariate time series that exhibit $\\alpha$-mixing behavior with exponential decay coefficients. This condition is denoted as Assumption 1.e. While the constant $2c$ is present in the exponent of the exponential function, it plays a non-essential role in our theoretical framework. We include it solely for the sake of convenience during verification.\n\nThe $\\alpha$-mixing condition with exponential decay as specified in Assumption 1.e  is a commonly held assumption in time series analysis. A broad spectrum of multivariate time series satisfies this condition,  including linear/nonlinear VAR models [e.g. 1],   a comprehensive class of GARCH models [e.g. 2], and various Markov processes [e.g. 3]. To further elaborate,  consider the\n$p$-dimensional stationary VAR(1) model:\n        $$X_t = A X_{t-1} + \\epsilon_t,$$\nwhere $A$ is the $p \\times p$ transition matrix whose spectral norm satisfying $||A|| \\in (0,1)$ and the innovations $\\epsilon_t$ are i.i.d. Gaussian vectors. Denote $\\Sigma = cov(X_1)$, and let  $\\lambda_{\\max}$ and $\\lambda_{\\min}$   be  the largest and smallest eigenvalues of $\\Sigma$. Then by Theorem 3.1 in [4], we have that for any $k \\geq 0$, the $\\alpha$-mixing coefficient of the time series $X_t$ satisfying\n        \\begin{align*}\n            \\alpha_k \\leq \\sqrt{\\frac{\\lambda_{\\max}}{\\lambda_{\\min}}}\\|A\\|^k \\leq e^{-C\\log(1/\\|A\\|)k},\n        \\end{align*}\n        where $C > 0$ is some constant depending only on $\\sqrt{\\lambda_{\\max}/\\lambda_{\\min}}$. In this example, the constant $C\\log(1/\\|A\\|)$ corresponds to the constant $2c$ in Assumption 1.e. \n\nEssentially, Assumption 1.e is useful to unlock several technical tools under temporal dependence, which include a Bernstein’s inequality [5], a moment inequality [see Proposition 2.5 in 6], maximal inequalities (see Section G.1) and a central limit theorem (see Section G.2). For instance, we utilize the moment inequality to bound the autocovariances of a dependence process with all lags by $\\alpha$-mixing coefficients, thereby demonstrating the existence of the long-run variance, which is the sum of all the autocovariances.\n\n**Minor issues**\n\nThank you for pointing these out.  The quantity $\\eta_k$ should be $\\hat{\\eta}_k$ and we have changed in our revision.  Regarding Line 131, we have edited to \"*we use $\\widehat{\\kappa}_k$ as bandwidth for the kernel density estimator in deriving $\\widetilde \\eta_k$*\".\n\n**Questions 1 & 2**\n\nThank you for pointing these out.  We have corrected them correspondingly.\n\n**Question 3**\n\nIn Theorem 2.a, you are  right that we do need the density function $f_{\\eta_k} $ to remain as a constant or converge asymptotically, we will include this condition in our revision. \n\nThe $\\alpha$-mixing condition is imposed on the data sequence, while $f_{\\eta_k}$ and $f_{\\eta_{k+1}}$ are population marginal density functions.\n\nFor  $F_{t, h_2}$, you are right that it is a typo.  For each change point $\\eta_k$, it should be $ F_{\\eta_k+t,h_2}$ and $ t<0$ corresponds to the time series before the change point $\\eta_k$.  With this correction, the distribution of $ \\widetilde \\eta_k$ does not depend on $f_0, f_1, f_2$ when $\\eta_k$ is far from 0.  To be more precise, as a byproduct of the limiting distribution in Theorem 2, $|\\widetilde \\eta_k -\\eta_k| =O_p(\\kappa_k^{-r/p-2})=o(\\Delta) $.  With high probability, the distribution of $ \\widetilde \\eta_k$, therefore, only depends on data in the interval $(\\eta_{k-1},\\eta_{k+1})$.\n\n**Question 4**\n\nWe appreciate the reviewer's observation.  In equation (7), the correct expression is indeed $\\kappa_k^{p/r-2}$.\n\n\n**References**\n\n[1] Eckhard Liebscher. Towards a unified approach for proving geometric ergodicity and mixing properties of nonlinear autoregressive processes. Journal of Time Series Analysis, 26(5):669–689, 2005.\n\n[2] Farid Boussama, Florian Fuchs, and Robert Stelzer. Stationarity and geometric ergodicity of bekk multivariate garch models. Stochastic Processes and their Applications, 121(10):2331–2360, 2011.\n\n[3] Kung-Sik Chan and Howell Tong. Chaos: a statistical perspective. Springer Science & Business Media, 2001.\n\n[4] Fang Han and Wei Biao Wu. Probability inequalities for high-dimensional time series under a triangular array framework. In Springer Handbook of Engineering Statistics, pages 849–863. Springer, 2023.\n\n[5] Florence Merlevède, Magda Peligrad, Emmanuel Rio, et al. Bernstein inequality and moderate deviations under strong mixing conditions. High dimensional probability V: the Luminy volume, 5:273–292, 2009.\n\n[6] Jianqing Fan and Qiwei Yao. Nonlinear time series: nonparametric and parametric methods. Springer Science & Business Media, 2008.\n"}, {"review_id": "8U7h5asQhB", "reviewer": "Reviewer_DHN6", "summary": "The authors consider offline change point detection for multi-variate data where there could be multiple change points (specifically changes in marginal distributions from one time step to the next).  The marginal densities of the underlying generative model are assumed to be smooth (specifically Hölder continuous, which includes Lipschitz continuous functions as a special case).  Building on a procedure using binary-segmentation search with kernel density estimation and consistency (Padilla et al (2021)), the authors analogously show consistency of change-point estimators for time-dependent data.   Furthermore, the authors derive limiting distributions of the change point estimators, both for the situation where the minimal jump size vanishes and for the situation where it remains constant.", "questions": "#### Questions\n- Assumption 3 and Theorem 1.  The $\\gamma_T$ does not explicitly show up in the Theorem 1 statement – which constants depend on $\\gamma_T$?  Given the assumption statement is only “arbitrarily slow diverging sequence” that seems to me impressive though almost too mild for anything other than statements about the limit $T \\to \\infty$ \n\n- For the theorems, is $\\Delta$ *required* to be growing as a function of $T$?  From the assumptions it looks like as long as the jump size is increasing fast enough, we could fix the location of $K$ changepoints and the assumptions would hold.  $\\kappa$ growing quickly should make detecting change points easier, but would there still need to be some minimum value of $\\Delta$?  \n\n- Contribution 1 (lines 84-87) –  The first major contribution claimed is the development of a novel algorithm and statement “To the best of our knowledge, we are the first to innovatively adapt SBS to a multivariate non-parametric change point model”.  From a (perhaps superficial) understanding, the Algorithm 1 proposed here seems to be an incremental adaptation of the procedure proposed in Padilla et al (2021), with the random binary segmentation used in the latter replaced with the recently proposed deterministic binary segmentation method SBS (Kovács et al. (2020)).  Perhaps the authors can add more discussion on why that change is not straightforward. \n\n\n#### Spelling, Grammar, etc.\n- Line 98 ‘A … estimators’\n- Theorem 2 statement – The formal statement could be shortened and I think easier to read if the notation (eg 188-189, 192- (7)) were introduced and discussed (add simple description of $P_k$’s formula when it is introduced) before the formal theorem statement.\n- For theorem 3, maybe $\\max_{1\\leq k \\leq K} \\dots$\n- line 291 ‘additional additional’\n- lines 291-293 – that is great you had further experiments; mention that earlier both in introduction and early on in Section 5.\n", "limitations": "It is fine (theoretical contribution)", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The authors study a challenging problem, non-parametric estimation of multiple changepoints.  Many prior works considered independent data and univariate, as well as stronger assumptions on the generative distributions.  \n\n- The authors prove consistency and analyze limiting distributions of the estimated change points.  (also this is with less knowledge (such as of optimal bandwidths) than Padilla et al (2021))\n\n- The authors include experiments and show its performance against baselines (designed for independent data). \n", "weaknesses": "My main concerns regard writing/presentation and discussion of related works.  I list a number of specific points below (many of which, by themselves, are fairly minor; list is not exhaustive).    \n\n#### Writing/presentation\n- There is no discussion on motivating the time-series model class considered ($\\alpha $-mixing sequences of random vectors with unknown marginal distributions), or even mention what types of common parametric classes belong to this class.   \n    -  The property of $\\alpha $-mixing is never explained formally let alone any discussion on what that implies at a high-level for the time-series.  There is a large literature on parametric time-series models – it would be valuable to mention example model classes that satisfy the assumptions here.\n    - What types of potential applications could this benefit (the time-series of which could plausibly be modeled with this class of models)?  Lines 20-24 mention application areas with time-series data, which is fine, but there should be more of a connection made between (some of) those applications and the specific problem  considered.  \n\n- Algorithm 1 and Section 3\n    - Provide discussion in the main text about the steps of Alg. 1 and intuition behind the design.  \n    - SBS is mentioned but is not described (a brief high-level description should suffice)\n    - There is notation used in the algorithm that is not mentioned in Section 3, explain what those are.  \n    - Include discussion about kernel properties in Section 3. Alg 1 uses kernel estimators implicitly in the line with $\\tilde{F}$, but up to this point no $\\mathcal{K}$ is defined or taken as input, or described in Section 3.\n    - (minor) Def 1 text $\\mathcal{K}$ has not been described (or listed in the def. set up)\n    - (4) and (5) give some verbal description of what this is doing (i.e. what is the intuition behind how you are improving the estimates, possibly with a line or two about the way(s) in which the MNSBS might give poor(er) estimates that leads to this design).  Presumably it has to do with that the choice of interval locations and sizing in MNSBS was using a simple pattern (oblivious to where the estimated change points would be).  Now you have ‘naturally defined’ intervals between the estimated change points (so size and location of the intervals are adapted to the initial estimates).\n    - line 124-125 talk about the choice of weights 9/10 and 1/10.  \n    - line 125 ‘an estimator of $\\kappa_k$’ I would suggest to add verbal descriptors for notation to remind the reader, such as ‘an estimator of the jump size $\\kappa_k$ at the $k$th change point’ and similarly for other notation.  Also, several lines later (to explain differences with prior work) there is mention that it corresponds to the ‘optimal bandwidth’ and ‘ we use $\\hat{\\kappa}_k$ as bandwidth for the kernel density estimator’ though in line 127 the bandwidth used is $h_1$ which depends on, but is not set equal to,  $\\hat{\\kappa}_k$.  A clarification of that part, and a couple sentences shortly after Definition 1 about the bandwidth $h$ to lead the reader to anticipate why $h_1$ in line 127 is good would help.\n    - while bandwidths are mentioned earlier (Def. 1 for instance) there is no discussion on bandwidth optimality.  Padilla et al (2021) have nice discussions regarding the choice of the bandwidth; I would suggest summarizing the key points to give intuition to the reader.\n    - line 132 ‘smaller interval’ – provide brief intuition about why the intervals here would be smaller than the\n\n\n#### Experiments\n- Include plots of the realizations of the time-series (to give intuition for how hard/easy the task is for those generative models)\n- Report the run-times for the proposed method and baselines.  With that also briefly describe the platform.\n- It would also be valuable to run experiments with independent data, to see whether the proposed method suffers compared to baselines designed for that setting specifically.  \n\n#### Related works\n- “Random Forests for Change Point Detection” by Londschien et al https://arxiv.org/abs/2205.04997  a multivariate nonparametric\nchange point detection methods for independent data (with an R package available) – in their experiments, their method generally was similar or better than ECP (the best baseline in your experiments) and had subquadratic time complexity\n\n- There are works non-parametrically estimating the location and size of structural breaks in non-parametric time-series regression models.  “Estimating change points in nonparametric time series regression models”  by Mohr and Selk (2020) https://doi.org/10.1007/s00362-020-01162-8,  “  Nonparametric inference on structural breaks “ by Delgado and Hidalgo (2000) https://doi.org/10.1016/S0304-4076(99)00052-4 among others.  Given some (at least superficial) relation in the problem considered and some methodological components collectively in that body of work and the present work, which extends methods and results for non-parametric changepoint detection for independent data (esp. Padilla et al (2021)) to allowing for dependencies, I think some discussion on the how the problems, methods, and results of changepoint detection for non-parametric   time-series regression relate would be helpful.  Xu et al (2022) is mentioned in lines 181-182, though what the paper actually studied and how it related was not described.  \n\n- A (brief) discussion of similarities and differences between the problem considered here and that of (non-parametric) online change point detection would be good. \n- One of the main contributions is characterizing the limiting distributions for multiple change point estimates in this non-parametric setting.  To my knowledge, that is novel for the specific problem considered.   However, similarities/differences in analyses and results (class of distributions that the limiting distribution belongs to) for past works that have investigated  limiting distributions of change point estimates in different by related settings are not well-discussed.  Unless I overlooked it, the only other work mentioned in the context of identifying limiting distribution of change point estimators is Xu et al (2022) in lines 181-182 and that was brief and implicit.\n    - There are prior works that studied limit distributions for non-parametric estimators for single change point detection with independent data (as well as works in the parametric setting), although from what I can tell under simpler changes (such as changes in the mean).  For example, “The Asymptotic Behavior of Some Nonparametric Change-Point Estimators” by Dumbgen (1991)  and Horváth and Kokoszka (1997) \"The effect of long-range dependence on change-point estimators.\"   Also “Optimal change-point estimation in time series” by Chan et al. 2021 for limiting distribution of single time-series change point (Bayes) estimators.   There is a recent work “On multiple structural breaks in distribution: An empirical characteristic function approach” by Fu et al (2022) that analyzes the limiting distributions for estimates of multiple change point. Also potentially relevant is “The asymptotic distribution of CUSUM estimator based on $\\alpha$-mixing sequences” by Gao et al (2022), though the analysis is for univariate non-negative $\\alpha$-mixing non-negative random variables.\n    - Limiting distributions of change point estimates have been studied in the parametric setting.  While those results do not diminish the significance of the results in this paper, there should be some mention and preferably a discussion on similarities/differences between the derived distributions. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive comments. We reply to your comments below, with corresponding edits in the revision.\n\n**Motivation on $\\alpha$-mixing sequences.**\n\nThank you for pointing this out. We have included the following in the revision.\n\nThe $\\alpha$-mixing condition with exponential decay as specified in Assumption 1.e  is a commonly held assumption in time series analysis. A broad spectrum of multivariate time series satisfies this condition,  including linear/nonlinear VAR models [e.g. 1],   a comprehensive class of GARCH models [e.g. 2], and various Markov processes [e.g. 3]. \n\n**Algorithm 1 and Section 3**\n\nThank you for all your valuable comments.  We will include all in the camera-ready version where an additional page is allowed.  For this revision, we refrain from the edits and just respond to your comments below. \n\n*Design and intuition of Algorithm 1*\n\nAlgorithm 1 has the SBS as its skeleton and a nonparametric version of the CUSUM statistics as its organs.  This is tailored to its nonparametric and potentially multiple change points nature.\n\n*High-level description of SBS*\n \nSBS is a multiscale version of a moving-window scanning method.  To conquer the potentially multiple change points with unknown spacing, instead of using a fixed window width, SBS uses a collection of window width choices, each of which is applied to a moving-window scanning.\n\n*Notation not used in Section 3*\n\nThanks for pointing it out. These pieces of notation are only local in the algorithm and are irrelevant to the rest of the paper.\n\n*Kernel function*\n\nFor Algorithm 1 to execute, there are no further theoretical assumptions needed for the kernel function.  In Definition 1, which is at the beginning of Section 3, the kernel function is already mentioned.  In the revision, we have specified that $\\mathcal{K}$ is a kernel function.\n\n*Equations (4) and (5), the choice of the weights and ``smaller interval''*\n\nGiven the consistency of the initial change point estimators procured from Algorithm 1, the interval $(\\widehat \\eta_{k-1},\\widehat \\eta_{k+1})$ is anticipated to contain merely one undetected change point. By conservatively trimming this interval to $(s_k, e_k) $, we can safely any change points previously detected within $ (\\widehat \\eta_{k-1},\\widehat \\eta_{k+1}) $. Consequently, the trimmed interval $ (s_k, e_k)$ is likely to contain only true change point $ \\eta_k$ with high probability. Due to the same reason,  our choice of weight in (4),  1/10, is a convenient choice.  Any constant weight between 0 and 1/2 would suffice.\n\n*The choice of the bandwidth and its optimality*\n\nInspired by Padilla et al. (2021), who proposed to use $O(\\kappa_k) $ as an optimal bandwidth in the context of Lipschitz densities, we adopt $h_1 = O(\\hat{\\kappa}_k^{1/r})$ as the bandwidth for our kernel density estimator.  This choice incorporates the broader scope of our work, which studies a more general degree of smoothness.  Notably, if the underlying density functions strictly adhere to the Lipschitz criterion and $r = 1$, our bandwidth selection aligns with that recommended by Padilla et al. (2021).\n\n**Experiments and Random forests**\n\nSee general rebuttal.\n\n**Related works**\n\n*Literature in nonparametric CP, online CP, and CP inference*\n\nThank you very much for bringing up these valuable suggestions.  Due to the limited time and space, we will defer the detailed comparisons along with corresponding edits to the revision.  \n\nIn this response, we briefly comment on the comparisons.\n\nIn the nonparametric change point literature, different kernel-based methods are adopted for change point localisation and testing.  Compared to the existing work, we follow the suit of using kernel-based CUSUM statistics but incorporate temporal dependence, which is rarely seen in the literature.  Most importantly, we are unaware of existing work on nonparametric change point inference, which is the main selling point of our paper.\n\nIn terms of online and offline CP comparisons, the core methodology is largely shared, but with different goals and performance measurements.  It is also unclear how to conduct inference in the online CP context.\n\nMost CP inference work focuses on fixed-dimensional parameters as well as lacks tracking of many model parameters. Xu et al.~(2022), in terms of style, is indeed the most closely related. but tackles high-dimensional linear regression, fundamentally distinct from our nonparametric density estimation.\n\n**$\\gamma_T$**\n\nIn Assumption 3, we require $\\kappa^2 \\Delta \\log^{-1}(T) T^{-p/(2r+p)} > \\gamma_T$, with $\\gamma_T$ diverging arbitrarily slow.  Although $\\gamma_T$ does not appear in the theorem statements explicitly, its impact can be seen.  For instance, Theorem 1 shows that, with large probability\n$$\n    |\\widehat{\\eta}_k - \\eta_k| \\lesssim \\kappa_k^{-2}T^{p/(2r+p)} \\log(T),\n$$\nwhich implies that \n$$\n    |\\widehat{\\eta}_k - \\eta_k|/\\Delta \\lesssim \\gamma_T^{-1} \\to 0, \n$$\nas $T$ diverges.  This regulates that each CP estimator is close to one and only one true CP.  This is the key to the success of deriving limiting distributions.\n\n**Conditions on $\\Delta$**\n\nFor $\\Delta$, we only require that the signal-to-noise ratio functions of $\\kappa$ and $\\Delta$ satisfy Assumptions 3 and 4.  In nonparametric literature, the $L_2$-norms of densities are assumed to be bounded, which leads to bounded $\\kappa$.\n\n**References**\n\n[1] Eckhard Liebscher. Towards a unified approach for proving geometric ergodicity and mixing properties of nonlinear autoregressive processes. Journal of Time Series Analysis, 26(5):669–689, 2005.\n\n[2] Farid Boussama, Florian Fuchs, and Robert Stelzer. Stationarity and geometric ergodicity of bekk multivariate garch models. Stochastic Processes and their Applications, 121(10):2331–2360, 2011.\n\n[3] Kung-Sik Chan and Howell Tong. Chaos: a statistical perspective. Springer Science & Business Media, 2001.\n"}, {"review_id": "DfbH3CE064", "reviewer": "Reviewer_TS3z", "summary": "The submission studies offline multivariate non-parametric change point detection. \nThe submission proposes a method for this task by combining 1) CUSUM estimator/statistic with 2) seeded intervals and 3) a refining procedure. The proposed estimator has 1) an improved error bound with weaker assumptions and 2) a limiting distribution for inference.  \nBoth empirical and theoretical results are provided supporting the advantages of the proposed method.  ", "questions": "The theoretical results seem to be a good improvement to the existing theory. However, I did not dig deep enough into the proof to evaluate the technical novelty. It is also unclear how much novelty and significance there is in the methodology. \n\n1. Would it be possible for authors to compare to an ablation method, which just combines CUSUM with SBS without the refining step? Or, is this ablation just the SBS method compared in the experiments? If so,  would it be authors to explain why this method (which is quite close to the proposed one just without the refining step) performs so badly? \n\n2. Would it possible for authors to provide stds of the experiment results to make sure that the proposed method really provides significant performance improvement?   \n\nAs a result, this submission is really on the borderline to me. I tend to accept the submission for the theoretical contributions. \n", "limitations": "NA", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The submission extends the existing theoretical results from the setting with Lipschitz smooth densities and temporal independence to the case with Hölder smooth densities with $\\alpha$ mixing, and also with a limiting distribution for the non-parametric case. \n\nThe writing is clear, with notations well defined.\n\n ", "weaknesses": "The submission is not presented in a very motivating way. For example, SBS appears abruptly without the background or reasoning. The methodology is also presented without explaining the motivations. \n\nThe methodological contribution of the proposed method is not clear. Combining CUSUM  and SBS is nothing new and also mentioned in the original SBS paper. As a result, the refining step in the proposed method is important for the methodological novelty of the submission. However, unfortunately, there are not enough discussions to motivate this refining step and emphasizing the novelty of this refining step. \n\nThe experiments can also be improved. The figures are hard to read since it is hard to match the competing methods to different colors. The advantages of the prosed method are not clearly demonstrated: some bars are very close to each other without showing significant advantages of the proposed method. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive comments. We reply to your comments below, with corresponding edits in the revision.\n\n**Intuition on algorithms**\n\nThank you for your valuable comments.  We will include all in the camera ready version where an additional page is allowed.  For this revision, we refrain from the edits and just respond to your comments below. \n\nOur method consists of a two-step procedure that incorporates the use of SBS, which is an approach for efficient change point detection in large-scale data.  SBS determines the best-split point for various search intervals in a greedy manner, forming a deterministic pre-computed set of search intervals. This deterministic nature of SBS allows for computational efficiency, achieving a total length for the search intervals that's linear, up to a logarithmic factor, irrespective of the number of change points. \n\nAlgorithm 1 is designed to yield consistent change point estimations with high probability. To further refine the precision of our final estimators, we apply a local refinement step. Notably, the interval \n\n$(\\hat \\eta_{k-1}, \\hat{\\eta}_{k+1})$ \n\nis anticipated to contain a single true change point, $\\eta_k$. To ensure that we can exclude any other change points within \n\n$(\\hat \\eta_{k-1}, \\hat{\\eta}_{k+1}),$\n\nwe trim this interval to $(s_k, e_k)$ as outlined in Equation (4). \n\n**Novelty**\n\nWe do not intend to claim much novelty from the algorithmic aspect.  The contributions of our paper lies in the general framework and the theoretical results.  To reiterate, the nonparametric change point inference with temporal dependence is first time studied in the literature.  On top of this, our refining step leads to optimal localisation errors.\n\n**Numerical results**\n\nWe will enhance the clarity of figures in the revision.  We would like to highlight that the strengths of our approach not only lie in the estimation precision, but also in its capability to produce statistically valid confidence intervals for these change points, a feature notably absent in many existing methodologies.\n\n**Methods without refining**\n\nThe refining step is necessary in two aspects.\n\nFirstly, the first step leads us to an adaptive bandwidth, tailored to each individual change point $\\eta_k$. The optimal bandwidth for each change point $\\eta_k$ is of order $O(\\kappa_k^{1/r})$, where $\\kappa_k$ represents the difference in density functions measured in the $L_2$ norm.  In a multiscale setting where the size of the changes can significantly vary across different change points, a non-adaptive choice of bandwidth parameter could lead to sub-optimal estimation errors for the change points. \n\nSecondly, the proof techniques we derive for the limiting distribution relies on the loss function to contain one and only one change point.  This is unachievable without a two-step method.  \n\n**Question 1**\nUsing only SBS and CUSUM in the first stage will not adaptively set bandwidths for each change point $\\eta_k$, risking errors; hence, a two-step procedure is advised for non-parametric time series change points.\n\n**Question 2**\nWe have included stds for all experiments we performed in our paper. See the extra page for more details about the stds in the extra simulations."}, {"review_id": "VXtiKcAacK", "reviewer": "Reviewer_rZb3", "summary": "This paper proposes algorithm to localize the multiple change points in nonparametric, short-term dependent time series. Assumptions required on the time series are certain mixing conditions and the smoothness in terms of density. The core idea of the algorithm is based on CUSUM and seeded binary segmentation. A two stage estimator is proposed, both of which are consistent, with the refined one in the second step achieving minimax rate under certain smoothness cases. Limiting distribution of the estimators are also given, which allows inference on the change points. Numerical results demonstrates the potential of the proposed method. ", "questions": "My questions have been raised in the \"weakness\" section.", "limitations": "While this work has nice and solid theoretical results, I wonder if practitioners can truly find the method useful, both due to the high computational complexity, and the somewhat complicated limiting distributions (which might prevent the full deployment of this methods' inference capability).", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This is a solid work with nice theoretical results. The problem tacked is a difficult one: localization of multiple change points in a nonparametric, locally dependent time series. The paper establishes not only consistent results, but also gives limiting distributions which can be beneficial for inference.  ", "weaknesses": "Although this is a solid work with nice theoretical results, I feel the related literature is not sufficiently reviewed which are crucial for determining the significance and novelty of this work. I only found in the last paragraph of page 1 a short summary of the existing literature, which I pasted below and raised some of my questions.\n\n\"Firstly, to the best of our knowledge, temporal dependence, which commonly appears in time series, has not been considered.\"\n-- I actually don't think this is true, as far as I know, there are a lot of existing work considering temporal dependence of time series (although I agree most of the exiting change point literature still assumes the iid setting). For example, I randomly searched for \"change point dependent process\" and there seems a bulk of related works, e.g., [1][2]. I believe the authors probably know better than I about these papers, and I wonder whether they could add some clarification on the difference between their setup and the one in this work.\n\n\"Secondly, there is no localization consistency result for data with the underlying densities being Hölder smooth with arbitrary degree of smoothness.\" -- it seems this is comparing to the work of Padilla et al. (2021) which only focuses on Lipschitz smooth densities, is my understanding correct?\n\n\"Lastly and most importantly, the limiting distributions of change point estimators and the asymptotic inference for change points have not been well studied.\" -- Do the authors specifically here refer to the papers on nonparametric, multiple change point literature? I believe the limiting distribution of change point estimators is one of the core questions in change point problems, and I am a bit surprised that the existing papers do not study them. Do the authors mean that most papers have been focusing on consistency results instead of the limiting distributions of change point estimators?\n\n[1] Ray, Bonnie K., and Ruey S. Tsay. \"Bayesian methods for change‐point detection in long‐range dependent processes.\" Journal of Time Series Analysis 23.6 (2002): 687-705.\n[2] Dehling, Herold, Aeneas Rooch, and Murad S. Taqqu. \"Non‐parametric change‐point tests for long‐range dependent data.\" Scandinavian Journal of Statistics 40.1 (2013): 153-173.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive comments. We reply to your comments below, with corresponding edits in the revision.\n\n**On temporal dependence**\n\nThank you for bringing these references to our attention.  We indeed overlooked them in the literature review stage.  We will modify our claim correspondingly.  We however believe the problems studied are still substantially different, especially that our focus is in the change point inference.\n\n**On the smoothness**\n\nYou are indeed right that we are comparing with Padilla et al.~(2021).\n\n**On the inference literature**\n\nYou are indeed right that we mean specifically nonparametric change point inference works.\n\n**More literature review**\n\nWe provide some more literature review here.  More will be included in the final version in due course.\n\nThere exist works such as [1] and [2], which estimate the location and size of structural breaks in nonparametric time series regression models. However, these studies do not address inference, time dependence or varying degrees of smoothness. Our proposed method is distinct in its adaptability to temporal dependence and its capability to handle a broad range of smoothness levels, providing a limiting distribution under these conditions.\n\nThere has been prior research on limiting distributions for nonparametric estimators but these mainly focus on single change point detection within independent data. For instance, [3] proposed a nonparametric change point detection and estimation model under the assumption of independent designs and the presence of a single change point. They also present a limiting distribution of the estimators. In contrast, our proposed method permits dependence with an arbitrary degree of smoothness and does not require prior knowledge of the number of change points. Furthermore, unlike [3], we analyse the optimality of our approach.\n\n[4] considers the estimation of change points in a mean detection problem where errors exhibit long-range dependence. Although they derive a limiting distribution of the estimator like our method, their model requires the knowledge of the number of change points, assuming a single change point exists. In contrast, our model does not make such assumptions.\n\nThe recent work by [5] investigates the limiting distributions of multiple change point estimators. However, their analysis differs from ours primarily in two areas. First, they assume the number of change points to be known for deriving their theoretical results and propose a data-driven choice for it. Unfortunately, there are no guarantees provided for the estimation of the number of change points when a sequential test is proposed. Their model only ensures the estimation of the number of change points when this quantity is upper-bounded by a finite integer, and the dependence of the estimators on the estimated break fractions is suppressed. The second significant difference is our method's achievement of an optimal rate for estimating the change points, eliminating the log term, whereas [5] does not present such a rate.\n\n[6] has also a derivation of limiting distribution for the CUSUM estimator based on $\\alpha$-mixing sequences. This work is different from our work as their analysis is for univariate non-negative α-mixing non-negative random variables.\n\nPast research has studied the limiting distributions of change point estimates in the parametric setting. A key difference here is our extension of these previous works from finite to infinite dimensions, i.e. in nonparametric problems.\n\n**Practical limitations**\n\nWe will make our code public available in due course.  Limiting distributions and their consequence of constructing confidence intervals, obtaining $p$-values are of high demands from practitioners.  In the more challenging case when the jump size vanishes, the limiting distributions are based on standard Brownian motions, echoing the universality.\n\n**References**\n\n[1] Mohr, M., & Selk, L. (2020). Estimating change points in nonparametric time series regression models. Statistical Papers, 61(4), 1437-1463.\n\n[2] Delgado, M. A., & Hidalgo, J. (2000). Nonparametric inference on structural breaks. Journal of Econometrics, 96(1), 113-144.\n\n[3] Dumbgen, L. (1991). The asymptotic behavior of some nonparametric change-point estimators. The Annals of Statistics, 1471-1495.\n\n[4] Horváth, L., & Kokoszka, P. (1997). The effect of long-range dependence on change-point estimators. Journal of Statistical Planning and Inference, 64(1), 57-81.\n\n[5] Fu, Z., Hong, Y., & Wang, X. (2023). On multiple structural breaks in distribution: An empirical characteristic function approach. Econometric Theory, 39(3), 534-581.\n\n[6] Gao, M., Ding, S., Wu, S., & Yang, W. (2022). The asymptotic distribution of CUSUM estimator based on α-mixing sequences. Communications in Statistics-Simulation and Computation, 51(10), 6101-6113."}, {"review_id": "nI0cRkpLan", "reviewer": "Reviewer_Ng5S", "summary": "This paper studies the problem of localization of multiple change points for offline multivariate time series. A non-parametric kernel-based CUSUM statistics is used together with the SBS algorithm. Moreover, a two-step estimation procedure is proposed where the initial estimate is further modified to the final estimate. The consistency result and the limiting distribution of the change point estimators are proved.  ", "questions": "In the current theoretical results, the consistency result is proved for the initial estimate, while the limiting distribution is provided for the final estimate. It would be better to also present the consistency result for the final estimate \\tilde{\\eta}, and to elaborate more on whether or not we still have such limiting distribution for the initial estimate. \nAnd it would be better if the author could illustrate in the numerical results the advantage of the second step (refined estimators), i.e., to show the improvement in estimation accuracy after using the refined estimator.\n\nIn the numerical results, the standard deviation of the misestimation rate (fig1) should be provided. The report of std can be very helpful here for comparing different methods since the average misestimation rate of MNSBS (blue) seems to be larger than or only slightly smaller than the other four baseline methods in various scenarios.\n\nIn the final estimate, line 126 on page 4, there is a typo in the subscript of F_{s_k,\\eta_k}, \\eta_k should not appear here since it is the unknown true change-point.\n\nIt would be better if the author could provide a pictorial illustration of the SBS algorithm.\n\n\n\n\n", "limitations": "Yes.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The main strength of the paper is its theoretical findings. Although the SBS and the kernel CUSUM statistics are both well-known, the theoretical results on the consistency and especially the limiting distribution under the setting of multivariate time series seem to be novel. ", "weaknesses": "The presentation can still be improved, and the numerical results is a bit limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your constructive comments. We reply to your comments below, with corresponding edits in the revision.\n\n**Weakness**\n\nThank you for your suggestions.  We will endeavor to improve our presentation and enhance numerical results in the revision.  For this response, extra simulations were conducted. We examined Scenario 1 with $p = 3$, $n \\in \\{150, 300\\}$, and $X_t$ as i.i.d.~$N(0_p, I_p)$. In Scenario 3, we added Random Forest for Change Points (RFCP) as a competitor. Runtime comparisons for methods with $p=3$, $n\\in{150, 300}$ are considered. All results are in Tables 1 and 2 (extra page).\n\n**Initial estimators**\n\nAs for the initial estimators, the consistency results are presented in Theorem 2, while the limiting distributions are not obtainable based on the current proof techniques.  The current proof relies on a uniform tightness property and the uniqueness of the minimizer in loss functions' population counterpart.  Neither of these two points holds for the initial estimator.  \n\nAs for the numerical results, in Scenario 1 (additional simulation), we compared our method with and without refinement. Results indicate MNSBS excels in change point localization, and refinement enhances performance, see Table 2 (extra page).\n\n**Report of standard deviations**\n\nWe have included stds for all experiments we performed in our paper. See the extra page for more details about the stds in the extra simulations.\n\n**$F_{s, \\eta_k}$**\n\nThank you for pointing out this typo, which we will edit in the revision.\n\n**Pictorial illustration of SBS**\n\nThank you for your suggestion.  We will refer to the plot in the original SBS paper in the revision.\n\n"}]

Author rebuttal:
We are grateful for your constructive comments. Taking advantage of the extra page submission we include the following extra simulations to help us to address each of the particular reviews.

**Independent data**

We examined Scenario 1 with $p = 3$, $n \in \{150, 300\}$, and $X_t$ as i.i.d.~$N(0_p, I_p)$. Results indicate MNSBS excels in change point localization, and refinement enhances performance, see Table 2.

**New competitor (Random Forest for change point)**

We compare the random forest change point (RFCP) method in [1] with our proposed method and the rest of the competitors, using Scenario 3 of our simulation studies. The results are summarized in Table 1, which shows that our MNSBS is generally outperformed.


**Runtime comparison**

We compared our method's runtime with others on a machine equipped with an Apple M2 chip (8-core CPU) for $p=3$ and $n\in\{150, 300\}$ in the independent setting.  Our method is comparable at $n=150$ but slower at $n=300$ due to CUSUM's computational demands. See Table 2 for more details.


**Plots**

We include a plot of realizations of the time series under the independent setting, in Figure 1. See the supplementary materials for a plot illustrating our considered real data.

**References**

[1] Malte Londschien, Peter Buhlmann, and Solt Kovacs. Random forests for change point detection. arXiv preprint arXiv:2205.04997, 2022


Meta-review: This paper studies the non-parametric offline change-point detection in multivariate time series, focusing on scenarios where probability density functions exhibit Hölder continuity. The authors suggest a two-stage process: segmenting the time series and then refining the identified change-points. They introduce a method integrating the CUSUM estimator, seeded intervals, and a refinement procedure and present enhanced error bounds. The paper extends prior work by demonstrating consistency in change-point estimators for time-dependent data. Additionally, the authors present the asymptotic distribution of estimated change-points and an algorithm to determine confidence intervals for each identified change-point. Empirical and theoretical evidence underscores the efficiency and advantages of the proposed methods. I believe the paper provides a valuable contribution to the literature.
Decision: Accept (poster)