Detection and Localization of Changes in Conditional Distributions
Authors: Lizhen Nie
Dan L Nicolae
Venue: NeurIPS 2022 Accept (2022)
Keywords: change point analysis
nonparametric
kernel methods

Abstract: We study the change point problem that considers alterations in the conditional distribution of an inferential target on a set of covariates. This paired data scenario is in contrast to the standard setting where a sequentially observed variable is analyzed for potential changes in the marginal distribution. We propose new methodology for solving this problem, by starting from a simpler task that analyzes changes in conditional expectation, and generalizing the tools developed for that task to conditional distributions. Large sample properties of the proposed statistics are derived. In empirical studies, we illustrate the performance of the proposed method against baselines adapted from existing tools. Two real data applications are presented to demonstrate its potential.

Detection and Localization of Changes in Conditional Distributions
Lizhen Nie Department of Statistics The University of Chicago Lizhen@uchicaao.. edu
Dan Nicolae
Department of Statistics
The University of Chicago nicolae@statistice. wch.icgo..ccoooo edu
Abstract
We study the change point problem that considers alterations in the conditional distribution of an inferential target on a set covariates. This paired data scenario is in contrast to the standard setting where sequentially observed variable is analyzed for potential changes in the marginal distribution. We propose new methodology for solving this problem, by starting from a simpler task that analyzes changes conditional expectation, and generalizing the tools developed for that task to conditional distributions. Large sample properties of the proposed statistics are derived. In empirical studies, we illustrate the performance of the proposed method against baselines adapted from existing tools. Two real data applications are presented to demonstrate its potential.
Introduction
The canonical setting for change point (CP) analysis is concerned with the detection and localization of changes in distributions of an independent sequence of observations (æe, / = 1,2, ... ,n.. In many applications, however, the observation at each index is pair (æ4, (24,))) with x, set of covariates and y a variable of special interest for inference. This setting incorporates many important problems in variety of scientific fields. For example, in finance, market data often affected by monetary policies. Thus, to more accurately predict stock prices using this additional information, one may to first understand whether and when the relationship between the target stock price and its associated covariates changes over time 377 environmental analysis 6 where one goal learn the relationship between the levels of pollutants and the number of weekly hospital admissions for circulatory and respiratory diseases, which is subject potential changes. Before considering more complex models that incorporate time-varying relationships, one needs to understand first whether the relationship indeed changes and if so, when.
Another example comes from
Despite the existence of many applied scenarios where the relationship between pair changes, change point methods specifically designed for paired data are quite scarce. As far as we know, existing methods assume univariate 's and aim changes in the conditional expectation of y given
x. Moreover, most papers [26] 24 25
3y 2 4
22
[5 40 [1] focus on the setting where the 4
conditional expectation of y depends linearly
The nonlinear and nonparametric cases, however, are relatively under-investigated. The existing methods mainly focus on two settings. One setting arises from challenges in econometric modelling, and they focus exclusively
the problem of testing
for structural stability of the conditional mean function over time series [49 (perhaps more) important problem of localizing the change point(s) is not investigated. Another setting assumes that the covariate 's are fixed, univariate, and form an equi-spaced sequence lying in interval [0,
46 while the
32 A detailed review of these related works is included in Section
We are interested here in the case where both x and y can lie in a general space (not necessarily the real line), and aim identify changes in the whole conditional distribution, rather than only
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

the mean. This is of significance to many applications: for example, in finance, it is beneficial to understand changes in the volatility level of stock prices. In this paper we want to both identify and localize such changes. We will assume a random design setting. Contrary to the fixed design setting in [31 32 we assume that the covariate Tt are randomly drawn from some unknown distribution. This assumption is realistic and meaningful. Consider the example where sociologist wants to investigate whether the relationship between education and income has changed over years. A survey is distributed every year, which records both the covariate (education level) and the response (income). In this case, the covariates cannot be fixed (as one cannot determine who will take the survey).
In the random design setting, one could argue that, if we combine and y and treat them as one variable, sufficiently powerful change point method that can handle any change in distribution should perform well. This approach is straightforward to implement: however, it targets changes in the joint distribution p(æ,, y) = p(r)p(y x), while we are only interested in changes in the conditional distribution p(u x)) An intriguing question is whether method especially designed for change points in this conditional distribution will have better performance than those for joint distributions. As the first work to study these general changes, we answer this question by: (i) developing a mathematical formulation of this new problem and establishing its connection to a simplified task; (ii) investigating the advantages and limitations of applying existing methods to this new problem; and (iii) proposing new methodology.
2 Related Work
This section reviews relevant CP literature on paired observations. Please refer to REL || for reviews on the canonical (unpaired) setting. More related work is discussed in the Appendix Section B
Linear models. Most publications on paired data assume that the conditional expectation of y is 24
15 I and attention has been focused on analyzing changes in the slope. Recently, generalizations of this classic linear model have been investigated. One strand of work focuses on analyzing changes in high dimensional linear regression
linear combination of x [26.
25
39
models
30 42 50,
while others include seasonality and/or correlated errors K
Structural stability test for conditional expectation. Several econometries papers focus on testing the stability of the conditional mean function using nonparametrie methods. Their task is simpler, as only testing is concerned, and moreover, only changes in the conditional expectation are considered. Direct generalizations of their methods to our problem are difficult, but they provide good starting point.
40
19
46
consider univariate Tt, yt 's where x; forms an equi-spaced sequence, and they focus on changes in E[y, sx)]. Furthermore, 32 assumes that direction of the change is known. We note that, although the fixed and random design settings seem similar, method which works well for one might not even work for another. Intuitively, the fixed
Nonparametric methods for fixed design. 31 32
design setting can be formulated as finding break points in an otherwise continuous/smooth curve, which is different from our focus.
Trend filtering. The recently proposed trend filtering 27
also targets estimating conditional expectation in fixed design. By solving penalized least squares optimization problem, the estimated curve exhibit the structure of a piecewise polynomial function 4X the form of which is more restrictive than [3] 32)
48
511
Bayesian approaches. Bayesian methods also used for CP problems. The most relevant are based on the Gaussian process change point analysis [4] i 2X), which aims to detect change(s) in the mean and/or the covariance function of the Gaussian process regression. In order to derive a tractable posterior, Gaussian prior is usually placed on the mean functions. The parametric formulations needed for Gaussian process CP approaches make them more restrictive than the methods proposed in this work.
3 Problem Statement
Task I: Analyzing changes in conditional distributions. The ultimate goal is to analyze changes
in the conditional distribution for a paired sequence ((1,, yt). = 1. n) where x, E X' and Ve y' with (X, d) semi-metric space and y general space. We assume that the covariates iid
It Fx. while the conditional distribution of y given o, might go through an abrupt change.
2

---

Denote (fii:= E 2), 1.,fF1jx (|x== E } to be two sets of conditional distributions that differ on at least one X. Our task is to investigate:
(Detection) Testing the null hypothesis
Ho :
Yr De PYIX=E
t ,.,,...,,7
against the alternative
Vt Iz ~ F r|x==
1= 1,2, ,
HA:
30" E (0, s.t.
yt
Iz TIx==
1 7* 1,
where n" = [mp"]. Here [a] denotes the least integer great than or equal to (Localization) When rejecting Ho, obtain an estimator 7 of.
Task II: Analyzing changes in conditional expectations. We start by considering simpler task. Assume that y C IR (i.e., response is scalar), (X' d) is Euclidean space, and the focus is only on potential changes of the conditional mean of y/ given Tt. For this simpler task. suppose fo. f1 X + R are two unknown functions with fo * f1, 's are independent random variables with E(e,) and Var(e,) oc, and the & 's are independent from F, 's Let F Ft be two probability distributions (not necessarily equal). We are concerned with the following two questions:
(Detection) Testing the null hypothesis
Ho Ve fo( t a Fo,
t= 1,2,
against the alternative
Ut fo(xt) ta,
a FO
t= 1=1,2 1.2,..... TT'
HA: 30* E (0,1) s.t.
(2)
1 fi(z)) t & Fl.
t= +1....
where T" = [np*].
(Localization) When rejecting Ho. obtain estimator i of r*
We will adapt methods developed for Task II to solve Task We start with some notations.
Notations. Denote 4 convergence in distribution. For sequence of real-valued random variables X,, and a sequence of real numbers Un, denote X, = Oas. (u,,) if P(EC < co, 3n, Vm > n, [X, Cuum)= 1, and X,, Oas. (u,) P(VC > 0, 3n, Vm > [xml Cu,) 1. Denote Op. 1p RP as the vector of all 0's and I's, respectively. Denote Ip E IRXP as the identity matrix and as the indicator function. Denote ||2 Euclidean norm.
4 Methodology
4.1
Solution to Task II
The underlying idea is to transform the CP problem with paired data into the more amenable CP problem for univariate observations. The best performing transformation, in our view, is reported next. Another possible transformation that has good performance, but without theoretical guarantees, is reported in Appendix
The proposed transformation relies on proper estimates of the conditional expectation; specifically, we consider the Nadaraya-Walee Watson (NW) estimator:
f(.) Ei lix (hx² d(-,
Σi kx (hx d(., x())
with x kernel function, and hx = h1 sequence satisfying h, -+ as n -> oc.
Introducing the transformed sequence Å.. A natural direction is find sequence {Á,,
1, 2, ... n such that Å, E [0, oo) represents the degree of difference in conditional expectation for data before and data after t. One such sequence is shown in Figure II where we see that without
3

---

null
alternative
60 02 04 BE 05
02 04
1000
200
800
800
1900
Figure Plot of transformed sequence Å, against L. In both panels Fx F (0, 1), fo(x) = x. n = 1000. For the right panel f1(x) = =², F. N(0, 1), 7" = 500. Y-axis in both panels are set to the same scale for comparison.
change point (left panel), Á, is flat across all i's; and with change point (right panel), max, Å, will be larger and the maximizer arg max, Å, roughly lies around f = r- Thus, inference for the CP problem could be based on the location and magnitude of the maximum among {á.].
To define the sequence } used for inference (and for generating Figure I we introduce a natural sequence, {år}. as follows. At each t, consider dividing the data into two parts: before t (i.c., {(o,, v)), 1, ... t}} and after (i.e.. {(2,, y:),
1, t + -.. ,n)). We fit two functions,
the left-side estimate ft(,,-); using only data {(T,, M:), = =:0=:,22 2, ,',, and the right-side estimate (t,-,, using only data (( Us), = + 1. 2, ,.. Intuitively, with some mild conditions, under the null, we have for all t's
(4)
= f+(t,-) =0 fo(.),
while under the alternative, when =T, f~(4-) ~ So(), ( ni), --(1,-)-44
fo() - fi().
However, under the alternative, when + =", both ft,,, and ft, will roughly lie between fo and fi. and we expect the 'difference' between (t,-) and (t...) -) to be smaller than that between fo and fi. Formally, we define the 'difference" as
Ex-Fx(-(L,X) f+ (1,
which is different from the usual I2 or la distance between functions by emphasizing on their difference the value of æ's is with larger density in Fx. practice we do not know Fx, and the expectation in (5) can be approximated by
n If (t, 2t, =: A..
(6)
or more economically, a Monte Carlo estimate using fewer particles (n' < n)
(t, x (n') If
f 2t)
iid where x/ ~
Fx and Fx = (1/n) Ei 6z, with o₂ the delta measure centered at x.
Standardizing A, into á,, It can be shown that the sequence Å satisfies the trend shown in Figure 1 (see Theorem C.1 in the Appendix), i.e., when sample size is sufficiently large, Å, R O under the null, while under the alternative, Â increases with before change point, while decreases after it. While directly using Å, for solving CP problems might work, better choice for localization is to consider properly standardized version (denoted, Â which is comparable across all t's under the null. This requires more careful analysis of the limiting distribution of f- under the null. Proposition C,2 in the Appendix shows that under the null, with some mild assumptions, for any t= [np] with [po, P1] and any fixed œ, +
Vnei (hx If (t, f+ xt, 2 N (0. c2(x)/(((1 p)]),
(8)
where a (hx). ), (2(2) are constants depending on hx and x separately. With Definition result immediately suggests setting
Å, = [t(n Ae/n|Ae.
(9)

---

such that Å follows the same limiting distribution (and thus is comparable) for all t's. Notice that the standardized sequence ( 1, ... n.} is used in drawing Figure I The proposed method named KCE (Kernel-based change point analysis for Conditional Expectations).
Before summarizing KCE and introducing the detailed algorithm, we would like to discuss the similarity of of the proposed method with the well established CUSUM [5], the definition ( and standardization in ? might look very similar to those in CUSUM. Indeed, one can think of KCE as a generalization of CUSUM where the estimate for mean replaced by the estimate for conditional expectations; a similar argument holds for the KCD algorithm proposed next to detect changes in conditional distributions.
Summary of algorithm and complexity. As mentioned before, after calculating Å, the estimator S PI < 1 We have
for r" is set to f = arg mä A. Here nop = [n n. [npi ]. < pre-determined in order prevent selecting not discussed yet the detection step. Traditionally, either analytic formulas or re-sate sampling methods have been used to calculate p-values for CP detection. The (limiting) distribution of max Á, is complicated, and we adopt resampling -based based method obtain p-values. The complete procedure to solve Task II is summarized in Algorithm [I Note that we store some pre-computed matrices for speed-up, and the overall time complexity is (n²) and space complexity (n?)).
few samples near the ends
Algorithm 1 KCE to solve task II (conditional expectation change)
input: observations {a,, Ve )=)) significance level a, parameters fi, Ri.
output: estimated change point location f. (f = implics no significant change point)
pre-compute:
Kx [kx (Nix "ey'd mT)) R=*"
2. _=x= where Asj- = [Kx]a
3. B E gaxx, where B:j- = [Kx]:88--
4. C, where [C] = Ei=1 Bu.
for = ne. no + 1,... ni do
for 1,2,
do
calculate (2,, = [C]u/Au and f4 (m., = ([C),n (C]a)/(Ain Au).
end for calculate Å. [t(n t)/nl Σ If (1, (t,,
end
detection: obtain p-value for max Stgnn å, using permutations or bootstrap. localization: p-value a, estimate f arg maNnoSisa Âg else,
4.2 Solution to Task
So far we have derived the solution for Task II, the conditional expectation change point problem. We show next how to adapt the proposed procedure to solve the more difficult Task I, the conditional distribution change point problem.
Generalizing Å., Note that the NW estimator is linear y. and thus A, can be written as
w(t,
2,,
(t,,,,,,,, (10)
1-=1
where denote for simplicity xx (h ((.. Ex (i,5), then
kx((.) n kx((rr El Rx (.r)
Ax(i))
if St.
w(t, x4,,
1 E Em Ex (1.7) En - kx(
kx(i
(x()
ifi,j> L
(11)
En Et (t.r) Erero (dr))
kx (i.))
(x
ifi t.
5

---

We replace the inner product terms Vedly in Equation TT by by (36. My) (a standard kernel technique), yielding
Â
wit i.J, I3)ky Uz) +
i,, (Hi.
w-=1
1,j=41
itt,i,,,, By Uz)-
(12)
Some notes on 12 x4 = c for all i's (which can be viewed as the canonical umpaired setting, observing only (yx ).-)), A, equals the maximum mean discrepancy [16] for testing equality between the distribution of {ys. t and that of {u,, i t). In the paired setting, the weights ( depend also on the value of 2, 's: the more similar Ti, 2.j are, the larger the weight on (Ui, Vÿ) becomes.
Standardizing Åc. Similar to Task II. one can show under some mild assumptions that A, defined in
(12) has the same trend as in Figure (see Theorem 5.1 in Section 5p. Standardizing Å, however, is much less obvious. We defer the rigorous derivation to Theorem SJJ in the following section. Here give some high-level intuition: standardization essentially relies variance of Áe; when deriving it, because of the randomness of x; s, a uniform bound that does not depend on ang is needed. To
establish uniform bounds, need to put some assumptions the space of æ's. In order to regulate the complexity of the space X, we introduce Kolmogorov's 's entropy, defined as:
4x (c) = log (N, (2)),
with N₂ (X) the minimal number of open balls of radius e in order to cover X. Now we are ready to present the standardization for Å, Recall the re-scaling factor in 2 is 1/t + 1/(n - (), and we expect the re-scaling factor here to be inflated by the Kolmogorov entropy of space X' in some way. The simplest case is when X' is compact and X has a density with respect to the Lebesgue measure: we can still set Å, using definition
D); and in the general case we suggest setting
Å [(1/t)xxx (lo) + (1/(n t) ((log(n - (n)
Intuitively, for any fixed t [mp], the larger the sample size, the smaller the variance of Á, and thus the smaller the rescaling factor. For any fixed n and f, the more complex the space X, the larger uxx, and the larger the rescaling factor. A more detailed derivation on this standardization is included the next section.
Summary of algorithm and complexity. The complete procedure for solving task I is summarized
in Algorithm Again store some pre-computed matrices to save time. The time complexity of Algorithm is O(n²) and space complexity O(n²). The proposed method is named KCD (Kernel-based change point analysis for Conditional Distributions).
Theory
We present only some relevant theory for Task I, and defer the rest to Section C and all proofs to Section D in the Appendix. From the Moore-Aronszajn theorem, the function ky ky-(y,) satisfies ky (y. y (y. Jy (y'. :) where H is a reproducing kernel Hilbert space, as long as kernel ky (t.)) is positive-definite Denote the conditional mean maps before and after change point
fo(x) By-ro [bY (Y.+)). [1()) Ey x (ky (Y,+),
in the sense that (f((z), ky 1) Ey p (Y, for
Define the norm || ||x
V6s and let
A = Ex-rx [x(x) (x)
(13)
Let B (2, -2' xx x) < h} be closed ball in X. We need the following assumptions to establish the asymptotic behavior of ÂŞ
Assumption 1 (small ball probability of Fx). There exists non-decreasing function ml-)) such that
3 3C1,, C2), Vz e X, Ve > 0, 0 Cim (E) < <P B (z,e)) S C2m (E) 80.

---

Algorithm 2 KCD to solve Task (conditional distribution change)
imput: observations ((2,, Mr 2=) significance level a. parameters no. Rt
output: estimated change point location f. (f = implies no significant change point) pre-compute:
Kx (hx d(a,, 2,) [))] R"x* Ky [ky U;)) E R=*x.
A R=*= where Aij = -i=1(KX] (xx]a.
3. B (1/n)KxX3) Xx R
C R**" where Cip Bτ Kyle--
for = ne, no + 1.... A1 do calculate Q E R2xx with
Cn
if st, if 4, 1
Audii
Gjj
Qi5
AueAAa
Ast)²
if > t.
(A,n A Ay1))
calculate Á, = Ei,=1 Qu + Eij-et1 Qis -2 Ej Qu- end for
detection: obtain p-value for maX Å using permutations bootstrap. localization: if p-value estimate f arg maXmoSeSm; Å,; else, estimate f
Assumption (Lipschitz continuity of forfi) fi). Functions fo. f1 satisfy
3C3 < oo, 3b 0, Vz, x' E X, [lff(z)) - fo(x'))l + +1f((()) - fi(z')ll S Ozd (z, z')
Assumption 3 (regularization kernel kx). (-) is nonnegative, bounded and Lipschitz contin- uousffuctiinn with support [0, 1). and if kx (1) = 0 the following conditions have to be satisfied
(C4, Cg) -0 < C₁ < (t) < Cs <, Vt [0,1),
3G. > 0, 3mo0 > 0, An No 8 m(()dd > Conm(1)--
Assumption 4 (regularization on kernel ky)) ky (:..) is a continuous, symmetric, positive definite, and uniformly bounded kernel.
Assumption (topological complexity of X). The topological complexity (x(()), together with the small ball probability function m(-), satisfy
3C, > 0, 3mo > 0, Vn < No. 0 S m' (11) < C7,
3no: Vn > no, (log n)² [mm]hh]] < dx (ogn/(mm(hh)) < nm(h)//logn),
33 > 1, Σm= exp {11 3) d (logn)/n)) < xc.
Remark 5.1. Assumptions 1-5 are technical, but, in general, they are mild and not that restrictive. For example, when the space X' is compact, and kx by are Gaussian kernels, Assumption 3 are all satisfied. If we further assume is Euclidean and the random variable x has a density function, Assumption is satisfied. Assumption 2 is probably the most abstract, but it essentially states that the conditional distribution p(Y
x) changes smoothly in the sense that p(Y | X 200 is close to
p(Y X =x1) when To is close to x1.
Theorem 5.1. Suppose Assumptions
3
4
hold.
(1) Under the null, for any t [np] with p [Po. P1).
Â Ous. (Itm(h))- x/x (t- log t) [[n ()m(h))]1 ((n ((n-1) 1) loggn -t)).
(2) Under the alternative, for any 1. [np]] with p E (0o, ((n,p")((( (p(ppp,
(1-p-)//( p)2 if upps S p* where 8(p) - (0")2/p²
Á - = Ous. (1), =os
(14)
fppp
4 are standard conditions for obtaining pointwise convergence of NW estimators. Assumption comes from AT3 and regulates the topological struciure of the infinite dimensional space X. An example of m, x)x if is a compact subset of TRP, vx((e) = Ollog(1/e) and, if X has density with respect the Lebesgue measure, m(h) O(h)).
Remark 5.2. Assumptions
7

---

Table 1: Mean + standard error the I localization error | r'| for each method, summarized over 20 simulations. The best performing method is marked in bold font.
Experiment A.
(b) Experiment B.
(1())
11
Dy
Dxy
KCE
31
Dy
Pxy
KCD
can(a]
239.1=41.11
98.42:00.6
120.61:8844
322.5=55.6
1.710.3
2.720.5
2,1=0.33
10r
89.0112.6
2.720.5
36107.
1.3:02
240.1=4 47.3
70.2:28.8
285.22:4.2
297.75-46.1
9.1199
118866
2.9:0.5
4,9-1.3
2²
cos(s)
284.7=44.0
107.9-25.2
46.1+13.7
146.8243.7
38,4+11.7
48115
3.0:0.9
12.6+300
mmxx(0.1
311.7=44.3
10.244.3
6.42166
26206
18:0.3
56.6+222.9
257,4=4.22
36+12
1.9
97.5.12.0
63.0114.0
66117
3.110.88
201137
407.8=40.5
369.61.3.1
10.9129
3.1:0.6
1-3
249.2=34.7
1044399
1399:388
5,0133
Remark 5.3. Theorem 5.1 shows that as long as A + O, Ås satisfies the trend in Figure 1 Thus, performance of KCD crucially affected by A which is characterized in the next proposition.
Proposition 1. Under Assumption 2 if ky is characteristic kernel, and F x # Fylx== for at least one I E supp((Fx) ), i.e., the support of Fx, we have A + O.
Remark 5.4. Proposition shows that A acts discrepancy measure between two sets of condi- tional distributions, { X I. E supp( (xxp(Fx)3 and { X= E suppl(Px)] It is reminiscent of the maximal conditional mean discrepancy (MCMD) between two conditional distributions in (36]. Actually, A equals the expected value of squared MCMD between F F with X Fx. detailed discussion on the link with [3 is included in the Appendix (Section
6 Experimental Results
This section investigates performance in synthetic data. We report representative results on different forms of X, ) and types of changes, with additional results included in the Appendix.
Baselines. We consider three baselines: one existing (the fixed design CP method (II), and two adapted from existing abrupt CP methods for unpaired data (denoted by Dxy and Dy). Dxy applies an existing CP method by treating (ze: Vt)) as the observation, while Dy does so by discarding 2t- There are numerous CP methods to choose from, and we use S introduced in [33 which unifies many existing nonparametric CP methods and can viewed exactly as Algorithm 2 by setting Tt = c. For KCD (or KCE), Dxy, Dy, we set (u) exp(-u²) and the choice of byy depends on y. For the method in [3 there are restrictions on the kernel function that are violated by the RBF kernel. So, following 31. we set k(u) 1.5(1 =²) < < 1). All bandwidths used for all methods are tuned among Sn = {0.001, 0.01, 0.1, 1, 10} on 10 independently generated data sets.
metrics. We report the I localization error f - r"I and the empirical power, both averaged over 20 independent simulations. Power is calculated under significance level a 0.05 with value determined by 500 bootstraps.
Evaluation
Experiment A: y IR, Fly | xr] changes. We first consider task II, and we Fx N(0, 1), Fo = F1 (0, 1), n = 1000, p² 0.7, T" = 700, Po = 0.05, P1 = 0.95. Before the change point, we set fo(z) = 2, while after change point, we investigate different options for ft. For Dy, set (y, y') = exp(-lly 1/
for DXY, set kz x²) exp(-1|z =2 /2/h2) with 2, (ze, V. KCE uses Algorithm D with d(-,.) .) set Euclidean distance.
Localization comparisons are reported in Table la Note that the method in B1 and Dy perform not so well, which is expected as the former is designed for equi-spaced covariates, and the latter only looks at the marginal distribution of yr: Dxy performs better, but is still slightly worse than KCE. KCE performs quite well, obtaining the most accurate estimator in most settings. Power comparisons are similar and are included in Table Sa in the Appendix.
Experiment B: R. p(y x) changes. Here we investigate task I. We start with E IR, Ely does not change, yet pyy r) changes. Specifically, we consider the model
(oo(z)) vi (Iz)e,, t> 1'
t t
y/
where 6, iid (0, 1), no(x)) x and we consider different forms of v1 's's. We set Fx NNO0, 1), 1000, p² 0.7, 7" 700, Pa 0.05, P1 0.95. KCD uses Algorithm 2 with ky (v. v²)
8

---

expf-1ly 3 3 and d(:,-) ) set to Euclidean distance. Dy uses ky (v. y/) = expl-10 y //12/h5)) and Dxy uses (z expí-12 2112/h3) with & (t).
Localization results are summarized in Table 1b We observe that KCD performs the best among all alternatives, followed by Dxy. Dy, and lastly K1
Experiment C: general y E J/, p(y x) changes. In the most general setting, let us consider task with E IP with p 5, Fx = N (O,: and different forms of ) When designing experiments, we deliberately let the conditional mean of y given x to be invariant. We set kiz (2, x') <x("x''(a ky (y: (y.r/ v/) for Dxy, and by used in methods is tailored to each V. 311 is not applicable to non-scalar yı 's and is thus omitted. The other settings identical to Experiment B. The forms of V's that are considered include:
(1) = IR, and we set y1 ~ (0, . Ip) before change point, and yt ~ N(B' Ip) with B [81, B2, ....... [t] and 33 = 0.1 x xix À x 1p with different magnitude after the change point. We set ky.y.. y/) exp(-ly by -- }.
(2) y = RPxp, Denote Y, as the element in the i-th row and j-th column of Y. Before change point, set Y X,X, where X,, X, denote the element in the ith and jth coordinate of X, while after change point, we set it to different quantity. We set by (Y, Y') exp( I - 1 /h²) with Frobenius norm || ||..
(3) y/ P. the set of all univariate normal distributions with variance 1. Each datum y, is m = 10 observations independently drawn from N ()((F)), 1). We consider u(x;;) 1 with different magnitude X X's. We set ky Y") exp(-11y l / h with 2- W-sssessteinn metric| Ilw.
Table 2: Mean + standard error of the I localization error f -"1 for each method, summarized over 20 simulations in experiment C. The best performing method is marked in bold font.
y
HA
Dy
Dyy
KCD
R²
A 1.0
7.1422.
15.1444.4
6.742..1
11.7.13.9
9,4.322
11.6.13.3
RP
A 0.6
65.14222.
69.84428.7
38.2+12.9
RPEP
Y/ X
10.41228
10.2.12.8
5.7
Y6 (X,)"X
10,243.3
15.544.11
10.24-299
Bexp
Yg (lm(X)
1.900.7
5,9114
1,4:0066
P
A= 1.0
51.6.177.6
5.111.7
4.0.
P A =.8
25.2449.
7,01177
9,24:2.3
P
113.7=35.7
104.7431.7
49.1=112.3
Results for experiment C are summarized in Table 2 In general, KCD (or KCE) outperform DXY and Dy, often with significant improvements. This observation is consistent across all experiments. The trade-off is that KCD takes O(n²) time complexity, while KCE, Dxy, Dy take only O(n²).
7 Real Data Applications
UK stock index. The Financial Times Stock Exchange 100 Index (FTSE 100) share index of the 100 largest companies listed on the London Stock Exchange. We investigate potential changes in UK economy from January August 2016, a period containing the Brexit vote (06/23/2016). Following [20 21], we consider changes in the daily value of FTSE 100 stock index, compared to two "reference" time series: the NYSE Composite Index and the NIKKEI 225 stock index. The NYSE Composite is stock market index of all common stocks listed on the New York Stock Exchange, and the NIKKEI 225 stock market index for the Tokyo Stock Exchange operating the Japanese Yen. They are used here as indicators for the US and Japan economies. Following [20]. we divide the closing value of each index by its corresponding daily Euro exchange rates. All data are downloaded from https://www.nn.. www.wwwwwwwwttc..cooom
Data from the three stock indices are shown in Figure 2a
The processed dataset contains n 157 sample points, with y E E the FTSE 100 divided by Euro exchange rate to British pound, and x E R² the vector consisting of NYSE and NIKKEI 225 divided by Euro exchange rate to US dollars and Japanese Yen, respectively. Using KCD with cxpf-lly 3/1l2/h3 }.
(u) = expf-o -u2 d(x,x') = || - x' 2'112, h li 0.1 and no 5, = n 5, we identify 06/23/2016 as the change point, with a p- value < 0.01. At the significance level 0.05, we conclude that there is a change point, which consistent with 20 21), and the estimated change point is also the date of the Brexit vote.
0.1, (y.')))

---

Rescaled Stock indices (EUR)
Changes in Discount and Interest Rates
Indlox
rale
diacoutt race (oovarann))
NROKEI
NYSE
(rosprnee)
01/0612016 022460115 D4TT02201 250022016
(a) UK stock index example. The red dashed vertical (b) Market interest rates example. The red dashed line represents the date of the Brexit vote.
vertical line represents the CP estimated by 4
Figure 2: Real data examples. In both panels, the blue dotted line represents f estimated using KCD.
Market interest rates. We consider changes in the relationship between market interest and discount rates over 1973-1989. The discount rate is the rate "at which the Federal Reserve System (Fed) lends and is set by the Fed" Market interest rates usually respond to changes in discount rates, but the differentials in response might be different, reflecting different conditions of the economy. We examine the response of market interest rates to discount rates over time. The data we use are collected by [T0, where the yields of three-month T-bills are treated market interest rates.
The data set contains n 56 sample points, with covariate x R the discount rate, response y E R the market interest rate. We note that discount rates are changed irregularly, thus the sequence is not equi-spaced in terms of time. The sequence {(T,, 31)) 1 is visualized in Figure 2b
Using KCD with lx (x) exp( - d(x, x²) | x'112. h 0.1, ky =xxp 2'12/h2, ), h2 0.1 and no 5, n 5, we estimate change point 08/17/1979. This identical to the estimation in
( ly
[. and is close to the October 1979 change in the Fed's operating procedures (4) However, the p-value is 0.25, indicating that the change not significant. This similar to the result obtained using the vanilla symmetric response model in 4 but different from that using adjusted or asymmetric response model
8 Conclusion and Discussion
To identify changes in the conditional distribution of paired sequences, we investigate several approaches, some based existing algorithms and one novel. Our nonparametric method applies to general covariates and responses, and outperforms existing methods. Future directions of research include relaxing distributional assumptions, and improving the O(n") time complexity of Algorithm 2 As suggested by one reviewer, the question detecting changes in conditional distributions with multiple gradual change points is also important but unresolved. A rigorous treatment of this generalized setup requires nontrivial modifications of the method and a re-formulation of the mathematical set-up, but we provide simple windowing approach possible solution for multiple change points in the Appendix. There are no foreseeable negative societal impacts this work.
References
[1] Samaneh Aminikhanghahi and Diane J Cook. A survey of methods for time series change point detection. Knowledge and information systems, 51(():339-36) 2017.
[2] Donald WK Andrews, Inpyo Lee, and Werner Ploberger. Optimal changepoint tests for normal linear regression. Journal of Econometrics, 70(1):9-38, 1996.
[3] Laura Aspirot, Karine Bertin, and Gonzalo Perera. Asymptotic normality of the nadaraya- watson estimator for nonstationary functional data and applications telecommunication.. Journal of Nonparametric Statistics, 21(5):535-551, 2009. 2009.
[4] Jushan Bai. Estimation of change point in multiple regression models. Review of Economics and Statistics, 79(4):551-563, 1997.
[5] Hermanus Josephus Bierens. The nadaraya-watson kernel regression function estimator. 1988.
[6] Zongwu Cai, Jianqing Fan, and Runze Li. Efficient estimation and inferences for varying- coefficient models. Journal of the American Statistical Association, (5(451):888-90)) 2000.
10

---

[7] Edoardo Caldarelli, Philippe Wenk, Stefan Bauer, and Andreas Krause. Adaptive gaussian process change point detection. In International Conference on Machine Learning, pages 2542-2571. PMLR, 2022.
[8] Hao Chen and Nancy Zhang. Graph-based Shange-point detection. The Annals of Statisties, 43(1):139-176, 2015.
[9] Paromita Dubey and Mans-Georg Müller. Fréchet change-point detection. The Annals of Statistics, 48(6):331223335, 2020.
[10] Michael Dueker. The response of market interest rates to discount rate changes. Review, 74, 1992.
[11] SR Esterby, and AH El- El-SSharawi. Inference about the point of change in regression model. Journal of the Royal Statistical Society: Series C (Applied Statistics), 30(3):277-288 1981.
[12] Frédéric Ferraty, Ali Laksaci, Amel Tadj, and Philippe Vieu. Rate of uniform consistency for nonparametric estimates with functional variables. Journal of Statistical planning and inference, 140(2)33353322 2010.
[13] Frédéric Ferraty, Ali Laksaci, Amel Tadj, and Philippe Vieu, Kernel regression with functional response. Electronic Journal of Statistics, 5:159-171, 2011.
[14] Frédéric Ferraty and Philippe Vieu. Nonparametrie functional data analysis: theory and practice, volume 76. Springer, 2006.
[15] Edit Gombay. Change detection in linear regression with time series errors. Canadian Journal of Statistics, 38(1):65-77)) 10055-79,00 2010.
[16] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):22-773, 2012.
[17] Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5, 2009.
[18] Bruce E Hansen. Uniform convergence rates for kernel estimation with dependent data. Econo- metric Theory, 24(3):726-743 2008
[19] Javier Hidalgo. A nonparametric conditional moment test for structural stability. Econometric Theory, 11(4):671-69) 1995.
[20] Zdeněk Hlávka and Marie Hušková. Doubly paired change-point analysis. In Workshop on Stochastic Models, Statistics and their Application, pages 143-155. Springer, 2019.
[21] Zdeněk Hlávka, Marie Hušková, and Simos G Meintanis. Change-point methods for multivariate time-series: paired vectorial observations. Statistical Papers, 61(4):1551-1333 2020.
[22] Steven A Julious, Inference and estimation in a changepoint regression problem. Journal of the
Royal Statistical Society: Series (The Statistician), 50(1):51-61, 2001.
[23] Hossein Keshavarz, Clayton Scott, and XuanI Nguyen. Optimal change point detection in gaussian processes. Journal of Statistical Planning and Inference, 193:151-178, 2018.
[24] Hyune-Ju Kim. Likelihood ratio and cumulative sum tests for a change-point linear regression. Journal of Multivariate Analysis, 51(1):54-70, 1994.
[25] Hyune-Ju Kim and Lujing Cai. Robustness of the likelihood ratio test for a change in simple linear regression. Journal of the American Statistical Association, 88(423):664-877 1993.
(26) Hyune-Ju Kim and David Siegmund. The likelihood ratio test for change-point in simple linear regression. Biometrika, (6(3))409-4233 1989.

---

[27] Seung-Jean Kim. Kwangmoo Koh, Stephen Boyd, and Dimitry Gorinevsky. & trend filtering. SIAM review, 51(2):339-360 2009.
[28] Mare Lavielle. Detection of multiple changes sequence of dependent variables. Stochastic Processes and their applications, 83(1):79-102, 1999.
[29] Sangyeol Lee and Siyun Park. The cusum of squares test for scale changes in infinite order moving average processes. Scandinavian Journal of Statistics, 28(4))625 644, 2001.
[30] Sokhae Lee, Myung Hwan Seo, and Youngki Shin. The lasso for high dimensional regression with a possible change point. Journal of the Royal Statistical Society: Series B (Statistical Methodology), (8(1):193-211 2016.
[31] Clive R Loader. Change point estimation using nonparametrie regression. The Annals of Statistics, 2464):1667-1675 1996.
[32] Hans-Georg Muller. Change-poiin in nonparametric regression analysis. The Annals of Statistics, 20(2):737-761, 1992.
[33] Lizhen Nie and Dan L Nicolae. Weighted-graphh-base change point detection. arXiv preprint arXiv:2103.00 026800, 2021.
[34] Yue S Niu, Ning Hao, and Heping Zhang. Multiple change-poin detection: a selective overview. Statistical Science, pages 611-623, 2016.
[35] Ewan S Page. Continuous inspection schemes. Biometrika, 41(1/2): 1954.
[36] Junhyung Park and Krikamol Muandet. A measure-theoretic approach kernel conditional mean embeddings. Advances in Neural Information Processing Systems, 33:21247-21259, 2020.
[37] Pierre Perron. The great crash, the oil price shock, and the unit root hypothesis. Econometrica: journal of the Econometric Society, pages 1361-1401, 1989.
[38] Werner Ploberger and Walter Krämer. The cusum test with ols residuals. Econometrica: Journal of the Econometric Society, pages 27 1-285, 1992.
[39] Werner Ploberger, Walter Krämer, and Karl Kontrus. new test for structural stability in the linear regression model. Journal of Econometrics, 40(2):307-318, 1989.
[40] Richard E Quandt. The estimation of the parameters of linear regression system obeying two separate regimes. Journal of the american statistical association, 53(284))873-888) 1958.
[41] Yong Ren, Jun Zhu, Jialian Li, and Yucen Luo. Conditional generative moment-matching networks. Advances Neural Information Processing Systems, 29, 2016.
[42] Alessandro Rinaldo, Daren Wang, Qin Wen, Rebecca Willett, and Yi Yu. Localizing changes in high-dimensional regression models. In International Conference on Artificial Intelligence and Statistics, pages 2089-2097. PMLR, 2021.
[43] Michael W Robbins, Colin M Gallagher, and Robert Lund. A general regression changepoint test for time series data. Journal of the American Statistical Association, 11 111(514)6770663, 2016.
[44] Yunus Saatçi, Ryan D Turner, and Carl Edward Rasmussen. Gaussian process change point models. In ICML, 2010.
[45] Eugene F Schuster. Joint asymptotic distribution of the estimated regression function finite number of distinct points. The Annals of Mathematical Statistics, pages 84-88, 1972.
[46] Liangjun Su and Halbert White. Testing structural change in partially linear models. Economet- ric Theory, 26(6):1761-188 2010.
[47] Liangjun Su and Zhijie Xiao. Testing structural change time-series nonparametrie regression models. Statistics and its Interface, 1(2):347, 2008.
12

---

[48] Ryan Tibshirani. Adaptive piecewise polynomial estimation via trend filtering, The Annals of Statistics, 42(1):285-323, 2014.
[49] Michael Vogt. Testing for structural change in time-varying nonparametric regression models. Econometric Theory, 31(4):811 11-899, 2015.
[50] Daren Wang, Zifeng Zhao, Kevin Z. Lin, and Rebecca Willett. Statistically and computationally efficient change point localization in regression settings. Journal of Machine Learning Research, 22(248):1 1-6, 2021.
[51] Yu-Xiang Wang, James Sharpnack, Alex Smola, and Ryan Tibshirani. Trend filtering graphs. In Artificial Intelligence and Statistics, pages 1042-1050. PMLR, 2015.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] to [Yes]. [No], or [N/A] . You are strongly encouraged to include a justification to your answer, either by referencing the appropriate section of your paper or providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ??.
Did you include the license the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answwes below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
g
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section g
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [Yes] The assump- tions are stated when the theory is presented.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] See supplemen-
tal material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [N/A]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material URL? [No]
13

---

(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "_jBUbG196xD", "reviewer": "Reviewer_cmHY", "summary": "The paper discusses a nonparametric change-point estimation methodology for change-point detection in paired sequence of observations. The author(s) propose a methodology based on kernel smoothing estimator to detect change-point in conditional expectation. Further, they generalized the methodology to change-point detection in conditional distribution via a novel kernel estimator. The asymptotic properties of the estimator has been derived. The synthetic results and real data analysis validate the performance of the proposed estimator.", "questions": "Following are my queries for the author(s)\n(A) In proposition 1 we require $\\Delta\\neq 0$, is it possible to have an asymptotic requirement i.e. as sample size n goes to infinity $\\Delta$ remain bounded above zero\n(B) Are we allowed to have discrete covariates x's in the settings author(s) proposed?\n(C) How is the empirical power calculated in the experiments in section 6?", "limitations": "Yes they adequately addressed the limitations and potential negative societal impact of their work", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "strengths\n\n(A) A novel change-point detection technique for paired sequence of observations\n(B) Smart use of NW eatimator for change in mean detection\n(C) novel theoretical results discussing the asymptotic properties of the proposed kernel change-point estimator for change detection in conditional distribution\n\nweaknesses\n(A) It wasn't clear in both algorithm 1 and 2 how one should make the choice of h_X or h_Y?\n(B) permutation/bootstrap procedure for p-value computation could be computationally quite expensive\n(C) standardized version of \\hat{Delta}_t is quite close to the CUSUM type statistic applied in change point detection. Some discussion would have been useful there.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "CsEUkqcV07V", "reviewer": "Reviewer_CXB3", "summary": "The paper presents a novel approach to detecting change points in conditional distributions, in particular for a setting relevant in machine learning where covariates are drawn iid from a fixed distribution, but the relation to the label changes. The idea is to derive a sequence of differences in conditional distribution, standardize this sequence, and perform a hypothesis test whether a change point exists for the maximum difference in an interval and report his point. For detecting a change in the conditional expectation, the difference of conditional expectation before and after this point is estimated using the kernel-based Nadaraya-Watson estimator. In order to estimate differences in conditional distributions, the paper proposes to generalize the estimator to take into account all pairwise similarities of labels (measured by a suitable kernel). The paper then proves correctness for their approach. The method is empirically evaluated on synthetic data and two real-world use cases.\n\nThe paper is sound, the presentation is clear, the method is novel and interesting, and well placed within the literature. I vote for acceptance.", "questions": "- regarding the UK Stock experiment: there seems to be a second changepoint coinciding with the Brexit vote announcement in February, 2016. Does your method detect this as well?", "limitations": "The method is currently used to detect a single change point in a time series. It would be great to empirically evaluate whether it can also be used to detect multiple change points (e.g., by a windowing approach), in particular on real-world use cases, e.g., detecting stock market bubbles in stock prices. It would also be great to see whether the method is able to detect less abrubt changes in practice (e.g., by shifting the mean gradually in a short time interval from the initial distribution to the final one).", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- tackles a problem relevant to the NeurIPS community\n- novel and sound method\n- solid theoretical analysis\n\nWeakness:\n- the real-world use-cases are a bit limited. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "6OMlzfxnXtf", "reviewer": "Reviewer_ggSY", "summary": "The paper studies a problem of detections and localizations of changes using the discrepancies between the Nadaraya-Watson estimator on data before and after a time. The paper develops the methods from scalar output cases to a general settings of outputs using kernel techniques.\nTo detect and localize the changes in data, a statistical test is proposed.\n", "questions": "- Let’s consider a case $t=1$, obviously ${\\hat{f}}(t, \\cdot)$ (\\hat{f}{-}, sorry Latex does not work here) is a not-so-good estimator because it uses one data point. On the other hand $\\hat{f}_{+}(t, \\cdot)$ can be a reasonable estimation as it uses $n-1$ data points.  I think the statistical test should rely on presumably perfect estimators. It seems like $\\tilde{\\Delta}_t$ may not reflect well on the statistical test when there is a big difference in the number of left-side and right-side data. Of course, the variance $c_2(x)/[\\rho (1-\\rho)]$ can describe this. Do you have any results showing that the method can perform well when $\\rho^*$ close $0$  or $1$ (i.e. $0.1$ or $0.9$)? \n\n- As stated in Remark C.2, I wonder how to pick $c_2(x)$ in practice? It can greatly affect p-values.\n", "limitations": "\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is well-written in general. It states the problem clearly, giving solutions for simple tasks then the genel ones.\n- The experiment results look interesting and promising. For example, in real world data, the method can detect change points which agree with actual events.\n\nWeakness:\n- Lack of discussion or emphasis on the transition from task I to task II. Specifically, it would be better to give an explanation as to why Kolmogorov’s entropy is involved. It might be technical but will provide a sense of directions for readers\n- Missing some references of CP detection with Bayesian approaches. I believe the covariate shift would be related to this line of research, especially there is work using kernel maximum mean discrepancy to tackle the covariate shift problems that resembles the approach in this paper.\n\nReference:\nGretton et al. 2008. Covariate Shift by Kernel Mean Matching. In Dataset Shift in Machine Learning.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "-Bq5pRuM8SG", "reviewer": "Reviewer_pAEt", "summary": "This paper addresses the problem of detecting abrupt changes in the conditional distribution of two (multivariate) random variables, using a finite time indexed data sample. The presented proposal entirely builds on a non-parametric approach. The presented approach is theoretically studied and some consistency results are provided. An empirical validation is also presented with both synthetic and real data sets. \n", "questions": "Could you please provide the reasons for the equation defining tilda delta at the beginning of page 6? \n\nHow realistic are the assumptions supporting your theoretical analysis? \n\nIs Theorem 5.1 of relevance in the context of finite data sets? \n", "limitations": "Mostly yes. The only issue is the set of assumptions made for the theoretical results. See my question about. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The presented approach is quite general and its implementation is simple. It does not assume the existence of a likelihood model, in opposition to many previous proposals. \n\n- Authors provide a non straightforward theoretical analysis of the consistency of the presented approach. \n\n- The presented methodology applies to multidimensional problems, which is a problem not widely addressed in the existing literature. \n\nWeaknesses:\n- Some parts of the approach are not derived using well-defined principles. For example, at the beginning of Page 6: “we suggest setting” the tilde delta variable according to some equation whose rationale is not discussed and it is hard to understand. \n\n- All the theoretical results established in this work are based on complex technical assumptions which are hard to interpret and, in many cases, impossible to verify. \n\n- A central issue in this kind of problems is “how big” should be the “abrupt change” to be possible for the proposed method to detect it from a finite data sample. This is not explicitly addressed in this work. \n\n- The computational complexity of the approach makes it impractical for many real-life applications. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept