Maximum Mean Discrepancy on Exponential Windows for Online Change Detection
Authors: Florian Kalinke
Marco Heyden
Georg Gntuni
Edouard Fouché
Klemens Böhm
Venue: Accepted by TMLR ()

Abstract: Detecting changes is of fundamental importance when analyzing data streams and has many applications, e.g., in predictive maintenance, fraud detection, or medicine. A principled approach to detect changes is to compare the distributions of observations within the stream to each other via hypothesis testing. Maximum mean discrepancy (MMD), a (semi-)metric on the space of probability distributions, provides powerful non-parametric two-sample tests on kernel-enriched domains. In particular, MMD is able to detect any disparity between distributions under mild conditions. However, classical MMD estimators suffer from a quadratic runtime complexity, which renders their direct use for change detection in data streams impractical. In this article, we propose a new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows. We prove that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and show empirically that it outperforms the state of the art on benchmark data streams.

Published in Transactions on Machine Learning Research (01/2025)
Maximum Mean Discrepancy on Exponential Windows for Online Change Detection
Florian Kalinke
kakiikeekkikkeekkil edu
Karlsruhe Institute of Technology, Germany
Marco Heyden
marco.heyden edu
Karlsruhe Institate of Technology, Germany
Georg Gntuni
g-gntuni@@gaill.com
Karlsruhe Institute of Technology, Germany
Edouard Fouché
edouard. edu
Karlsruhe Instätate of Technology, Germany
Klemens Böhm
klemens. boehm Skitt edu
Karlsruhe Institute of Technology, Germany
Reviewed on OpenReview: https: openreveew. net/ forum? id= 0GoIF9100s
Abstract
Detecting changes is of fundamental importance when analyzing data streams and has many applications, e.g., in predictive maintenance, fraud detection, medicine. A principled ap- proach to detect changes is to compare the distributions of observations within the stream to each other via hypothesis testing. Maximum mean discrepancy (MMD), a (semi-)metrit on the space probability distributions, provides powerful non porammmttiiicc two-sample tests on kernel-enricedd domains. In particular, MMD able detect any disparity between distri- butions under mild conditions. However, classical MMD estimators suffer from quadratic runtime complexity, which renders their direct use for change detection in data streams impractical. In this article, we propose a new change detection algorithm, called Maxi- mum Mcan Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows. We prove that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and show empirically that it outperforms the state of the art on benchmark data streams.
1 Introduction
Data streams are possibly infinite sequences of observations that arrive over time. They can have different sources: sensors in industrial settings, online transactions from financial institutions, click monitoring on websites, online feods, etc. Quickly detecting when change takes place can yield useful insights, for example, about machine failure, malicious financial transactions, changes in customer preferences, and public opinions.
A change occurs if the underlying distribution of the data stream changes call this moment change point (Gama) 2010); it is sometimes also referred to as concept drift. A principled and widely-used approach to detect changes is to use two-sample tests. The null hypothesis of such tests is that the data before and after the potential change point follow the same distribution. Tf the test rejects the hypothesis, one assumes that a change occurred.
certain point in time. We
One way construct these tests use the kernel-based maximum mean discrepancy (MMD; Smola et al. 2007 Gretton et al. 2012), which one can interpret as (semi-)metric the space of probability

---

Published in Transactions on Machine Learning Research (01/2025)
distributions | In the statistics literature, MMD is also known energy distance (Székely & Rizzo 2004 2005); see Sejdinovic et al. (2013) for the equivalence. MMD relies on the kernel mean embedding (Berlinet & Thna 2004| Ch. 4); it uses a kernel function to map a probability distribution to reproducing kernel Hilbert space (RKHS: Aronszaju 1950) and quantifics the discrepancy of the two distributions as their distance the RKHS. MMD is a metric if the kernel mean embedding is injective; the kernel then called characteristic (Fukumizu et al. 2008 Sriperumbudur et al., 2010). When using a characteristic kernel, the MMD bwo-sample test allows to distinguish any distributions given that their kernel mean embeddings exist, which is guaranteed under mild conditions.
Two-sample tests based on MMD are widely applicable, as there exist kernel functions for multitude of Euclidean and non-Euclidean domains, for example, strings (Watkins 1999 Cuturi & Vert 2005), graphs
Garrnner et al. 2003; Borgwardt et al., 2020), or time series (Cuturi 2011 Király & Oberhauser 2019)
Another benefit of kernel-based two-sample tests is their high power. While, for Euclidean data, it has been shown that the power of such tests generally decreascs in the high-dimensional setting (Ramdas et al. 2015)
recent results (Cheng & Xie 2024] establish that the power rather depends on the intrinsic dinensioonllit of the data. The intrinsic dimensionality is typically low in real-world settings that the kernel-based two-sample tests there do not suffer the curse of dimensionality.
Despite these benefits, a well-known bottleneck of MMD-based approaches is their computational complexity. When comparing the distributions two sets of data of sizes m and n, respectively, the computation of MMD with classical estimators is in (m² + n²), with a memory complexity in O (m + n). Naively computing MMD for each possible change point on data stream with m + n observations has a complexity in O (t3) for each new observation. These properties render the direct application of MMD to change detection in data streams impractical.
In this paper, we introduce Maximum Mean Discrepancy on Exponential Windows (MMDEW), change detection algorithm for data streams that solves the above bottleneck. Specifically, our contributions include the following.
Our main contribution is MMDEW, a change detector based on an efficient online approximation of MMD, When considering the entire history of obsorvations, the proposed method has memory requirement of O (logt) and runtime complexity of O (log" t) for each new observation. Otherwise, the algorithm has constant iuntime and memory requirements.
To achieve these complexities, we introduce a new data structure, which allows to approximate the quadratic time MMD in an online setting. We accomplish the speedup by introducing windows that store summaries of the observations seen so far, and by storing a sample of logarithmic size of the observations per window.
Our experiments on standard benchmark data sets show that MMDEW performs better than state-of-the- art change detectors on four out of the five tested data sets using the Fi-score. For the more challenging setting of short detection delays, the proposed algorithm is better on three out of six data sets
Outline. Section summarizes related work. Section roduces the definitions and Section presents the proposed algorithm. We detail the experiments in Section 5 Section 6 concludes. We include illustrative proofs in the main text but defer technical proofs, additional details, and additional experiments to the appendices.
2 Related work
Change detection is an unsupervised task that has received and still is receiving a lot of interest. The earliest approaches, for example, [Shewhart] [1925] Page (1954), originated from quality control and require strong parametric assumptions the pre and post-change distributions. More recent work in the parametric regime weakens these assumptions by allowing post-change distributions from parametric family with an unknown
1A function semimetric if it a metric but can zero for distinct. elements. 2Qur code available at htttps://åtthub com//Foossaa/mmdeet change- detector
2

---

Published in Transactions on Machine Learning Research (01/2025)
parameter (Lorden
1970
Siegmund & Venkatraman 1995] or by allowing any post-change distribution
Sparks 2000 Lorden & Pollak 2005
Abbasi & Haq 2019: Xie et al. 2023
In our setting, both the pre and post-change distribution are assumed to be unknown, which is a challenging setting that can be tackled with non- non-parametric approaches. We detail the approaches most related to our proposed method in the following and refer to Wang & Xie (2024) for a recent more extensive survey on parametric and non-parametric change detection methods.
A principled approach for comparing distributions in stream in non-parametric fashion is to use corresponding statistical test. ADWIN (Bifet & Gavaldå 2007) is a classic example but it is limited to univariate data and only detects changes in mean. ADWINK (Faithfull et al. 2019) alleviates the former by running one instance of ADWIN per feature and issues a change if a predefined number of the instances agree that a change occurred. Hence, the approach can only detect changes in the means of the marginal distributions and changes in higher moments or the covariance structure can not be detected. Still, the authors find that such an ensemble of univariate change detectors often outperfomm multivariate detectors. WATCH (Faber et al.) 2021) is recent approach that uses two-sample test based on the Wasserstein distance. However, the estimation of the Wasserstein distance requires density estimation, which is difficult for high-dimensional data (Scott 1991p. The method Dasu et al. (2009) is conceptually similar to our method, as it also relies on two-sample tests and is non-parametric, but it also requires density estimation.
In contrast, the computation of MMDD-ased two-sample tests does not become more difficult on high- dimensional data, which renders their usage for change detection on such data promising. We refer to Muandet et al. 2017] for general overview of kernel mean embeddings and MMD.
There exist methods to compute MMD in the streaming setting, for example, linear time tests (Gretton et al. 2012), but their statistical power is low. Zaremba et al. (2013) introduce B-tests, which have higher power However, both can not directly be used for change detection. Li et al. (019) enable the estimation of
MMD on data streams for change detection by introducing Scan B-statistics. Wei & Xie (2022) extend upon
their work by considering multiple Scan B-stalistics in parallel and introduce online kernel CUSUM. Another
method enabling the computation of MMD on data streams is NEWMA (Keriven al.
2020). which is based
random Fourier features (Rahimi & Rocht 2007 Sriperumbudur & Szabó, 2015). well-known kernel
approximation. NEWMA also allows detecting changes streaming data. Harchaoui & Cappé (2007) apply
kernel-based tests for offline change point detection on audio and brain--ommputer-interfaa data.
A conceptually different approach to find changes is using classifiers. D3 Goöüüaçik et al. 2019 maintains
two consecutive sliding windows and trains a classifier to distinguish their elements. It reports a change if the classifier performance, measured by AUC, drops below a threshold. Another recent algorithm is IBDD Ide Souza et al., 2021,, which scales well with the number of features.
In our experiments in the main text, we compare MMDEW to ADWINK, WATCH, Scan B-Statistics, NEWMA, D3, and IBDD as these allow change detection on multivariate streams (in Rd). These algorithms differ w.r.t. their runtime complexity, their theoretical properties, the data types that they can handle, and
the types of changes that they can detect. We summarize their main properties in Table 1 3 We consider the dimensionality as constant for the complexities where its influence is dominated by other terms and for approaches not restricted to Euclidean domains. In Appendix [.] and Appendix D.2 we collect additional experiments on synthetic data. In particular, we additionally compare the proposed method to online kernel CUSUM and to multivariate adaptations of the Moanerr-on-Misee change point model (CvM CPM; Ross & Adams 2012) and non- non-parametric Focus (Romano et al. 2023).
3 Definitions and background
This section defines our problem and recalls kernels, the mcan embedding, maximum mcan discrepancy, and two-sample testing.
3o We refer to their used implementation of the Wasscrstein distance computation and the discussion therein Mérigot 2011 Ch. 6). pm the number random Fourier features and & 'The complexity results from the matrix inversion of the logistic regression model, which has cubic runtime cost. dSize of the constructed q x p image.
3

---

Domain Published in Transactions on Machine Learning Research (01/2025)
Table 1: Comparison of change detectors. Complexity runtime complexity per new observation, ARL / MTD type of known results, domain data types, total number of obscrvations, d dimensionality window length block size, N number of windows.
(for Euclidean spaces), k parameter, W
Algorichm Complexity
ARL / MTD
ADWINK
O (dk log W) unknown "
empirical
Rd R
WATCH
empirical
Scan B
O (NW*)
analytical
topological
NEWMA
O (mdy)
analytical
E
D3
O (W3)e
none
g
IBDD
O (pq)
none
R
MMDEW
O (log") 1)
empirical
topological
Problem definition. Let (X, rx) be a topological space, B(xx) the Borel sigma-algebra induced by rx, and Mf (X) the set of probability measures on X meant w.r.t. the measurable space (X, B(Tx)). We consider data stream, that is, possibly infinite sequence of obscrvations, 21,22.-... Fty... for t 1.2....: and t, E X. Each 2y is generaled independently following some distribution D, E (x)). If there exists i such that for i < t" and 2 t" we have D, * Dj, then 1* is a change point, and our task is to detect it; in practice, Dj typically generates a range of i.i.d. observations. We note that these definitions place few assumptions on the type of data, that is, we only require the data to reside in a topological space.
Kernel mean embedding. Let H be a reproducing kernel Hilbert space (RKHS) on X, which means defined by (f) f(x) is bounded for all æ X' and X unique vector o(x) H such that for every f E H. it holds that f(x) da(f) = (f. o(z)). The function 6(x) is the reproducing kernel for x and also called feature map; it has the canonical form x + k(,,z), with the function k: X x X + R the reproducing kernel associated to H. With this kernel, it holds that k(;, 22)1,22) = (0(x1), o(x2)) = (k(-, 21), k(;,,z)) x22) for all 01, E X' (Steinwart & Christmann 2008). ) The mean
that the linear evaluation functional 8 : H
f E H. By the Riesz representation theorem (Reed & Simon 1972, there exists for each
embedding of probability measure E MM is the element ((P) H such that E Ex<P f(X) = (f. ((P) for all f E H. The mean embedding p(P) exists if measurable and bounded (Sriperumbudur et al. 2010 Prop. 2), which assume throughout the article.
Maximum mean discrepancy. MMD is defined by MMD(PP, Q) ||m((P) ((Q)ll where u(P), E H are the mean embeddings of P,, Q (x),,, respectively.
Let X ~ P, Y ~ Q and X', y' independent copies of X, Y, respectively. The squared population MMD Grreton et al. 2012 Lemma 6) then takes the form
MMD Q = E (AXXX ER((YY')) 2E (x(,Y)),
where the expectations taken w.r.t. to all sources of randomness. For observations Pm = (II.... Fmd iid. Lid. Q a biased estimator is obtained by replacing the population means with their
IP and Q (yr,..., .D) empirical counterparts
MMD² (P... 2n)
Σ
k (3)) + n f
k(yi. v;)
2
k( U;).
(1)
m i-==
mn
The runtime complexity of is in O (m² + n²). We will base our proposed approximation on (
Two-sample testing. To decide whether the value of MMD (p... @n indicates a significant difference between IP and Q one tests the null hypothesis Ho P Q versus its alternative Hi : PP + Q by defining acceptance region for a given level a E (0, 1), which takes the form MMD (Pmm. 2.) < <a. One rejects Ho if the test statistic exceeds the threshold. The level is a bound for the probability that the tests rejects Ho incorrectly (Casella & Berger 1990). Assuming that & is nonnegative and bounded by K > 0, that is,
4

---

Published in Transactions on Machine Learning Research (01/2025)
0 I k(x,y) K for all y, X, Gretton et al (2012 Corollary 9) provides the distribution-fre threshold for the case that both samples Pm and Qm have the same size (m = n) as
2K m
!
MMD
1
2 log a
(2)
Computing ( costs (1)). As the change detection setting requires the case that m # n, we extend their threshold accordingly in what follows.
4 Our proposed algorithm
We introduce MMDEW in three steps. We first exlend the threshold for the MMD two-sample test, to samples unequal sizes (Section 4.1). We then introduce our data structure that enables the efficient computation of MMD on data streams (Section 4.2 Last, we describe the complete algorithm in Scction 4.3.
4.1 Threshold for the hypothesis test
Given a sequence of observations (Ti up until time t our goal is to test the null hypothesis P Q for any two neighboring windows X Y (21....,.z)) {xs+1...... ......................... 2th, with 1.....tt. 1. Our following proposition extends Gretton et al. (2012 Theorem 8), which considers the case n, giving the distribution- free acceptance region for * (corresponding to the setting that one generally encounters in change detection). The proof is deferred to Appendix A.1
Proposition 1. Let P,Q (x)), P. (II,..., Fm) s.d. IP, @n = (un...., .u.. s.d.d Q. Assume that
0 0 k(x, w) < K for all x, y E X and t > 0. Then a hypothesis test of level at most a > 0 for P = Q has the acceptance region
MMD (P... Qn
+ n (1+ <(1+
K
loga-!) =: Ca-
Note that, when considering multiple possible change points, one needs to account for multiple testing in order to achieve overall level of size a. For example, may adjust ca through Bonferroni correction (ca ca/(t 1)) by dividing by the total number of tests.
To perform change detection with MMD, it is natural to consider the stopping time
T inf (1:
-.=-....11 [MMD (Pms Qa) 2 ca) cal=1
(3)
for m = t- the empirical measures Pm {II..... Lm) Qn = fim+ls.... ærh. and the brackets equal to one their argument true and zero otherwise (Graham et al.) 1994); we note that ca == <a (m, n) depends on the respective sizes of the subsamples considered. In other words, a change is indicated by the first time any MMD estimated across all splits exceeds its threshold. However, due to the quadratic runtime roquirements of MMD, the computation of (3) costs (t3) for cach new observation.
We now int roduce our novel data structure that allows considering multiple possible change points elficiently.
4.2 Proposed data structure
One common method to obtain good runtime complexity in change detection algorithms is to slice the data into windows of exponentially increasing sizes (Bifet & Gavaldà 2007). Recent observations are collected in smaller windows, and older observations are grouped into larger windows. This leads to a fine-grained change detection in the recent. past. and more coarse-grained change detection in the distant past.
Our new data structure adopts this concept and, at the same time, facilitates the computation of MMD. In what follows, we first describe the properties of the proposed data structure. Then, we show how to update the data structure and explain its for change detection.
5

---

Published in Transactions on Machine Learning Research (01/2025)
4.2.1 Properties
We use 2 as the basis for the exponential slicing. Then, after observing elements, the number of windows stored in the data structure corresponds to the number of ones in the binary representation of t. We may thus index the windows as .,.... Bo (in decreasing order), with the largest position being I = [log fj- A window does not exist if the binary representation of t at this position is zero.
If it exists, a window B, = (X,, xX,, XY,) at position s = 0...., stores 2 observations
X,
{
#1
(5)
x2
(3)
(4)
together with the summaries
2"
XX, = Σ ho
(5)
2
2'
2
XY,
Σ Σ
k
(z) I
(x)
(0t11)
(5) !1) 2
(6)
=1
=:XY211
where XX, E is the sum of the kernel & evaluated on all pairs of the window's 's own observations, and XY, stores a list of sums of the kernel evaluated on the window's own observations and the observations windows coming before H Storing list enables the efficient merging of windows, elaborated in Lemma 2 The length of the list XY, equals the number of windows having observations older than window B, and is at most llog, 4]- We use XY? to represent the entry in XY; that refers to the window Bj. Specifically, in XY stores the interaction of B, with B441; similarly, XY, stores its interaction with Bi.
Remark 1. Given a stream of data 21, 22,....,
4 corresponds to the mapping 1 ze, with =
Llogt_ BB, exists) +i, where the bracket is one if the argument true and zero otherwise (using Iverson's convention; Graham et al. 1994). A bucket Bj ewists if the j-th right-most digit in the binory exponsion of is 1.
We sunmarize two of the main properties of the data structure as lemmas. Lemma [ establishes that one can compute the value of MMD between two windows with constant complexity. The proof follows from
comparing (5) and 66) with [ Lemma 2 shows that windows can be merged with logarithmic runtime
complexity. These results provide our first steps towards efficiently computing MMD in a data stream.
{ (e+1)
(s+1)
Lemma Let Bu+1 and B, be any two neighboring windows with elements X
2241
(5) and X, = { zi ,..., } and sums as defined by and respectively. Then
2
MMD? (X++1,X.) (20+1)2 XXo+1 (2 XX, (20+1)
XY
with a computational complexity of O().
Lemma 2. Merging two windows Bs+1 and Bg into a new window B', such that B' stores and costs O t).
Besides showing the result, the proof of Lemma
Sanammaiilsttrann illustrates the steps that allow merging windows efficiently.
Note that the superseript (s) (() (2) indicates the corresponding window Bg-

---

Published in Transactions on Machine Learning Research (01/2025)
X₂ - (xr)
XN₂ R(nyaa))
XYo = 0
Bg
Xo {m)
(m))
(21.22)
XXp R(n.an)
XXor M(22.ar))
Merge
Ein k(a))))))
XYo
XYr (2))
XY; 8
Bo
Bg
X₁ = (xL))))
X, = (ax)
23
XX₃ Σ Mayay))
XXu kiss
XY;
XYa E Maness)
B,
Bu
Figure 1: Schematic representation of Example in bold face.
I For a given step, the proposed scheme stores the windows
Proof. For computing XX', we use the symmetry of l to obtain
2--1
2 2
XX'
=
Σ
k
(o+1)
((+11)
Σ
k
hi
(5+1)
ΣΣ
ho Te 1 (et!))
w- 1
j=1
2--1
2
Σ
k
x
(s+1)
Σ
k
k
x
(0+1)
(5)
XX,+1 + XX, 2XY;+1
which has runtime complexity in 0(1).
To compute XY', we note that B8+1 stores the list XY.+1 of kernel evaluations corresponding to all windows coming before it. The same holds for Bg, for which the list has one more element, XY which was used in All the elements XY, and s41 are sums and thus additive; it suffices to merge both lists by adding their values elem omitting XY*+1, and storing the result in XY'. As each list has most logt elements, merging them is in O (logt).
Specifically, the scheme facilitates the merging of windows of equal size, enabling us to establish the expo- nential structure outlined in the next section.
4.2.2 Insertion of observations
The structure is set up recursively. For each new observation, create a new window Bo, with XX, defined by (5) and XY, computed w.r.t. the already existing windows. If two windows have the same size, we merge them by Lemma) 2 which costs O (logt). This yields [logt] t] windows of exponentially increasing sizes.
We illustrate the scheme in the following Example I and the correspouding Figure [
Example To set up the structure, we start with the first observation 31 and create the first window Bo, with XXo as defined by (5) and XYo 0. When observing x2, we similarly create a new window Byy now also computing XY ) As Bu and By have the same size, merge them into Bt, computing XX, with 77. No previous window exisls that XY, = 0. We repeat this for all new observations, for example, for F3, one creates (a new) Bo, computing XX1 and XY, = [XY)), which results in two windows, B, and Bo-

---

Published in Transactions on Machine Learning Research (01/2025)
4.2.3 MMD computation and change detection
We now show that we can compute the MMD statistic at positions between windows with a runtime complexity of O(log)) Proposition Let Bi.....B.+1. B Bo be given list of windows with corresponding elements X = -,,... ,,, defined in For any split f1,...: 1}, the computation of
MMD Ü
x.XUU 1=0
i
Ux.)
(8)
1=++1
that is, the computation of MMD between the elements in windows coming before window B, and the elements windows coming after (and including) Ba, has runtime complerity of (logt) for < 8 < 1, with 8, E N.
Proof. To obtain 18 one recursively merges .s...., Bo to B' using Lemma 2 starting from the right, and similarly .,,..., BB+1 to Bj. One then obtains the statistic with Lemma and by setting XY = -' XY; at: that is, by summing all elements in the XY,, of By, This concludes the proof as the logarithmic complexily was already established.
The application of the presented data structure for change detection is as follows. For each new observation, we estimate MMD at any position between windows and compare to the threshold <, s (with Bonferroni correction) from Proposition We report a change when the value of MMD exccods the threshold. As there are at most. log/ windows, we have at most log/ positions. Computing MMD for position is in O (logt) by Proposition 2] and so the procedure has total runtime complexity of O (log2t+t) per insert operation, where the term linear in t results from computing XYo when inserting new observation. We may equivalently consider the proposed method as providing a more coarse coarse-grained estimate of (3) taking the form
MMD ( i==+1 i=0
(ÚX.Ú,, Ux,
2 ca =1
T' inf t: max s= Lyo.t
(9)
with the X,-s defined as in Proposition
While the data structure its current. form allows to obtain the precise values of in an incremental fashion, its runtime and memory complexity are O (t) for each new observation; these complexities are unsuitable for deploying the algorithm in the streaming setting. We reduce the runtime by subsampling within the windows, which we present logether with the complete algorithm in the following section.
4.3 MMDEW Algorithm
Our algorithm builds upon the data structure discussed previously. But, suggest that each window of size 29, 8 = 0, O,...,I, samples observations (of the total 2*), that is, a logarithmic amount, while keeping everything else as before.
In this section, we first analyze such subsampling and discuss its benefits. Afterwards, we present the complete algorithm. We refer to Appendix [A.2 for the proof of the following statement.
Proposition 3. With subsampling, the number of terms in the sum XX, for window position 1, 1 1 LEN
nxx. = 2/-1 (1-11+0 (log]
log, t 4).
with t = 2' the number of observations of Bi. The number of terms of XY! for windows of the same size, which occur prior to merging, is
xxy; 241 tog
RReall that Remark gives the precise value of I. Further, some split positions may not exist.
8

---

Published in Transactions on Machine Learning Research (01/2025)
Input: Data stream 21, .,, level
Output: Change points in 21, :.... detection times
1: windows +- 0
D List of windows
2: for each 3i E U2, do
3: Xo tT;
b Initialize Bo
XXo t- k((x,, x;i)
for each Bj windows do
6: XY t ExpreD k(1;, xx (())
7: Bo XXo,
8: windows + windows U Bo
9:
for each split s in windows = [Br.... Bot1, ,a...,,..BBu do
b Detect changes
if MMD (Uj=st) X,, Uio 2 6a then
10:
11:
print "Change at detected at time 1" windows <- B......P. Bo
12:
D Drop windows
13:
while two windows have the same size 2 do Merge windows following Lemma into B1+1 Store uniform sample of size 1 + in X141 of B
Maintain exponential structure
14:
15:
Algorithm 1: Proposed MMDEW change detection algorithm.
Remark 2.
The number of terms in the sums of (1] acts prozy for the quality of the estimate. It is optimal when no subsampling takes place; this number is O (t²). When subsampling logarithmie number of observations per window with our data structure (as we propose), one achieves polylogorithmic runtime and logarithmic memory complerily. At the same time, one achieves a better approrimation quality than naively sampling a logarithmic number of observations without the summary data structure. While such sampling would also yield a memory complerity of O (logt) t) when using the naive approach for change detection-the is, splilling the sample inlo two neighboring windows and computing MMD2 -the number of terms in (1) would be O (log²t). t). Proposition 3 shows that the summary data structure improves upon this by factor of approzimalely 1/2 for nxx, and factor of if logg for Tixx; (we neglect logarithmic and constant terms in the former due to their small contribution).
Algorithm)] 1 now summarizes the complete algorithm, with MMD in Line 10 referring to the computation of MMD as in Proposition 2 MMDEW stores only uniform sample of size 1, that is, of size logarithmic in the number of observations, while keeping the respective XX, and XY,, this approach, the number of samples in a window increases by one each time the window is merged, and the memory complexity logarithmic
0, O....,,, computed before. With
the number of observations. Note that one recovers the previous algorithm (Section 4.2.3) and therefore the precise value of (8] if one omits Line 15 Further, changes in Line 15 allow adjust the subsampling, for example, the user may defer the sampling until windows contain minimum number of observations, or choose different function to control the sample size.
The following example illustrates the procedure. Figure 2 expands upon Example and shows the evolution of the data structure upon observing I1,
T6 and when merging windows.
Example We assume that there is a stream of i.idd observations F F2..... Note that the i..d. assump- tion implies that there are no changes. MMDEW receives the first observation, 31 and creates window Bo storing 31, XXn k(21,21), and XY, 0. For the next observation, I22 it creates new window By storing X2, XX x(x2,x2), and XYo [k(z1, and detects no change. As Bo and By have the same size, MMDEW merges them into window Bı, storing sample of size log, 2 2 1, say, it stores In and discards Ø2, and compules XX1 = =(x1, + k(#2, x2) + 2k(x1, following ( As no previous window exists, the computation of XY, 1 is not required. We see that the number of terms XX1 equols four, while Proposition 3) .Nert, the algorithm observes 23 and creates a new window, Bo, storing F3, XX, k(zz, Ø3), and computing XY, the window coming before, that is, B1, that XY, (XY2)- In the next step, MMDEW receives F4, again creating new window By
B, stores only one observation (established
9

---

Published in Transactions on Machine Learning Research (01/2025)
= (n)
XX, a()
XYo
Bo
= (a)
Xir (mx)
Xi = {x)
Merge
21
XX. (()
XXo (2))
XX1
2 k(a)
XY6 0
XYr (R22)
XY1 =
Bo
B
X, = (x))
Xu = (an)
XX; - M(xaas))
XXa = 25)
XY, 0
XYu = (8()).2)))
X, - (x:)
N, - [ss]
Ny (Rc]
XX, - A(ren)))
XXa ((ts
XXy k(re:
XY, 0
XYo (8(2))25)))
XYw (k(rs.re), Mon.
Bi
Bg
Bg
X, [n)]
X (21)
X; (2))24))
Mergo
Mengo
XX,
Lig M(A)))))
XX, Lige Astis)
XX2 XX2 2XY,
XY,
XY. (M(r.ag) R(ss)R))
XY,
D
B
X, = (1).24)
Ng = [1s]
XX, XX, XXy y
XX, kt))
XY,
XY, (R(ervets) MeR
Xs = (4).24)
Xa = {1r)
No = (Zc)
Merge
XX, XX, XX2 2XYY
XX, k(2s.2s)
XXg: M(n,)a))
XY; 0
XY, (M(r). Rx).
XYp (MXr, anl R() Mo
B
Figure 2: Set up of data structure with subsampling upon inserting æ1,... IG. MMDEW stores the windows in bold face the end of the merge operations. Obscrvations 22 and my are not stored explicitly due to the sampling applied. Iy split into two lines for readability. See Example for detailed discussion.
The algorithm now recursively merges the windows, that is, Bo and By become By, and B1 and By then become B2. Upon receiving 2s, the algorithm creates new window Bo, storing 2s, the kernel evaluation k(#s, Rs), and the interaction of as with {I1, from B3. We conclude the enample with Pe, which leads to the creation of new window By, As in sleps 2 and 4, Bo and By will now be merged to oblain B
Algorithm I has a runtime cost of O (log" t) per insert operation and total memory complexity of O (log t). This allows it to scale to very large data streams. Nevertheless, if one strictly requires constant time and memory, one can simply limit the number of windows at the expense of detecting changes only up to certain time in the past. In the latter configuration, MMDEW fulfills the requirements for streaming algorithms laid out by Domingos Hulten 22003).
5 Experiments
This section showcases our approach on synthetic data (Section 5.1 and on streams derived from real-world classification tasks (Section 5.2p. We ran all experiments on a server running Ubuntu 20.04 with 124GB RAM, and 32 cores with 2GHz each.
10

---

Published in Transactions on Machine Learning Research (01/2025)
5.1 Synthetic data
To evaluate the average run length (ARL) and the mean time to detection (MTD) in controlled environment, we first conduct experiments on synthetic data, comparing MMDEW to the MMD estimate (1) as baseline m
We also compare the runtime of MMDEW to that of existing change detectors. For an in-depth comparison of the ARL and MTD tradc-off for kernel-based approaches with optimally chosen thresholds, see Appendix D.1 For a comparison of MMDEW with univariate approaches, we refer to Appendix D.2
ARL and MTD. The ART quantifies the expected number of observations processed before a change detector flags a change, assuming Ho holds. In the static setting, this corresponds to the type I error. Formally, for T corresponding to the stopping time captured by Algorithm that is, g with subsampling applied, we are interested in EH,
The error under the alternative (H1 holds) is captured by the expected detection delay (EDD), also called "mean time to detection (MTD)". Specifically, a change detector processes a stream that contains change at known observation K (K .t) and we want to know the delay until the change is reported -a
with P the pre-change distribution and Q * P the post-change distribution. In other words, the EDD quantifies how many samples of Q must be processed to flag a change after having observed k - 1 samples from IP. Note that k must be chosen large enough to allow MMD to capture the difference in P and Q, and we assume that the statistic does not exceed the treshold on the first k samples, In a static sctting, the EDD is comparable with the type II error. Note that existing litcrature (Xic & Siegmund, 2013 Wei & Xie 2022) usually considers a fixed threshold b for the stopping time for both ARL and EDD, while our threshold 6, depends the position of the split. considered. Experiments for fixed value of b are deferred to Appendix D.I
To approximate the ARL and the EDD in different scenarios, we simulate 5-dimensional data distributed according to the multivariate normal N (0, the uniform 4||11s, 1s), the Laplace (0, aIs), and a mixed distribution, respectively. The mixed distribution is taken to be N (0, Is) with probability 0.3 and A (0, (0,) with probability 0.7, where 1a denotes a vector of d ones and I, is the d-dimensional identity matrix. We set o 3.
To compute ARL, we consider 10,000 observations distributed according to either the uniform, the Laplace, or the mixed distribution. Hence, the data does not contain any changes. For MTD, we first run both algorithms on 512 (= 29) and 1024 (= 2d observations, respectively, leading to MMDEW summarizing the data in one window in both cases. These observations are distributed according to N (0. and then followed by either the uniform, Laplace, or mixed distribution. That is, we induce a change point at x = 513 (resp. K = 1025), and then count the number of observations processed from the new distribution until the algorithms report change.
Figure 3 collects the average results over 20 repetitions. The left plot shows that an increase in the level a E (0, 1) leads to a decrease in ARL. This is expected as the test becomes more sensitive, lcading to more false positives. The baseline achieves a higher ARL but at the cost of an increased runtime. The MTD plots (center and r.h.s.) mirror the ARL observation: The MTD decreases with increasing a. We further observe that the detection delay depends on the post-change distribution. The delay is comparably large when changing from the multivariate standard normal to the mixed distribution. This matches our intuition: the mixed distribution is relatively similar to the pre-change distribution, rendering it difficult to detect change between them. For larger values of a, that is, 2 0.2, MMDEW performs similarly to the baseline in all cases. Comparing the MTD when the change happens after 512 obscrvations to the MTD when the change happens after 1024 observations, the results show that more pre-change samples render the algorithms more sensitive to detecting changes, due to more samples improving the approximation of the mean embedding. Overall, the results on these synthetic streams indicate that MMDEW is (i) robust to the choice of a and (ii) that a has the expected influence on the behavior of the algorithm.
For computational reasons, we compute MMD as described in the discussion following Proposition use the distribution-fres bound of Proposition for both algorithms.
F fair comparison,
11

---

Published in Transactions on Machine Learning Research (01/2025)
Mixed
MMDEW Luplace
Uniform
Mixed
Luplace
10000
MTD (512)
400-
MTD (1024)
400-
200-
200
0-
02
0.4
0.6
0.8
1.0
0.0
0.2
0.6
0.6
1.0
0.0
0.2
0.4
0.6
Level
Figure 3: Average run length (ARL) and expected detection delay mean time to detection (EDD) / MTD) of MMDEW on synthetically generated data.
MMDEW
Adwink
IRDD
NEWMA
ScanB
WATCH
0.0020
10
0.0015
MMDEW
I Lincar lit loga1
10.
0.0010
Number of observations
Number of observations
Figure 4: Comparison of runtimes per insert operation (l.hs.) and least squares fit validating the theoretical runtime complexity of MMDEW w.r.t. the runtime observed in practice (r.h.s.).
Runtime. We now compare the runtime of MMDEW that of its contenders and additionally validate the runtimo guarantees that we derived analytically in Section
To this end, generate constant stream of 10 one-dimensional observations, that is, the observed st contains no change. Note that, while the dimensionality of the data affects the runtime depending on the used kernel, its inffuence is the same across all kernel-based algorithms, hence we limit our considorations to the univariate case.
Figure shows the average results over 10 runs. The left plot reveals that the fixed cost per insert of MMDEW is relatively large, as processing small number of observations requires comparably much time. However, the runtime does not increase by much with the number of observations. The figure also shows that the proposed algorithm's 's runtime is better than that of an alternate kernel-based method, Scan B-statistics, where we use a window size of w = 100 in the runtime experiments. For & > 0.05 - 105, MMDEW also oulperforms IBDD. Still, the other algorithms run faster than MMDEW but achieve a lower F₁ score in our later experiments.
The right plot of Figure verifies the analytically derived runtime of O (log* t) by fitting the corresponding curve ++ e logg t) to the measured data with the least squares method. The resulting mean squared error is approximately 10 16,6, which confirms the preceding asymptotic runtime analysis.
5.2 Real-world classification data
To obtain our change detoction quality estimates, we use well-known classification data sets and interpret them as streaming data." This is common the literature, for example, Faithfull et al. (2019): Faber el al.) 2021I, as only few dige-diie annotated change detection data sets are publicly available.
For each data set, first order the observations by their classes; change occurs if the class changes. To introduce variation into the order of change points, we randomly permute the order of the classes before each run but use the same permutation across all algorithms. For preprocessing, we apply min-max scaling to all data sets. Table 2 summarizes the data sets, where is the number of observations, is the data dimensionality, and #CP the number of change points.
While MMDEW is limited Fuclidean data, Euclidean data is the type of data frequently encountered in practice, and our experiments target this setting.
12

---

Published in Transactions on Machine Learning Research (01/2025)
Table Overview of data sets.
Data set
# #CPs
CIFARIO (Krzhhvssyy el al.
2009)
60,000
1,024
9
PashionMNIST [Xiao et al. 2017
70,000
784
9
Gas Vergara et al. 2012
13,910
128
5
HAR Anguila et al.
2013
10,299
561
MNNST (Deng 2012
70,000
784
MMDEW
AdwinK
D3
IBDD
NewMA
ScaniB
WATCH
1.0
K2.05 0.5
0.5
0.0
CIF
PMNIST
Go
HAR
MNET
CF
FMNIST
Gas
HAR
MNIST
CIP
FMNIST
Gis
HAR
MNNITT
1/4
Figure 5: Average Fi-score, precision and recall. The bars show the standard deviation over 10 permutations of the data.
We run a grid parameter optimization per data set and algorithm and report the best result w.r.t. the Fi-score. We note that such optimization is difficult to perform in practice-here approaches with fewer easy-to-set parameters but allows fair comparison. Table 3 in Appendix C lists all the parameters we tested. We note that the grid parameter optimization allowed us to obtain better Fi-scores than the heuristics proposed in Keriven et al. 20000) for NEWMA and Scan B-statistics.
typically prefers
We exclude the squared time estimator of MMD due to its prohibitive runtime. For kernel-based algorithums (MMDEW, NEWMA, and Scan B-statistics) we use the Gaussian kernel k(x,y) = exp (->llx - (7 > 0) and set y using the median heuristic (Garreau al. 2018) on the first 100 observations. The Gaussian kernel is universal (Steinwart & Christmann, 2008; Szabó & Sriperumbudur 2017 and allows, given enough data, to detect any change in distribution as universal kernel on compact domain is characteristic (Gretton) et al. 2012) Theorem 5). We also supply the first 100 observations to competitors requiring data to estimate further paramcters (IBDD, WATCH) upfront
Fi-score, precision, and recall. We compute the precision, the recall, and the Fi-score, which are common evaluate change detoction algorithms (Li et al., 2019 Keriven et al. 2020 van den Burg & Williams 2020 Faber al.] 2021). Specifically, for fixed AT E N>o, we proceed follows. change is detected, and there is actual change point within the AT previous time steps, consider it true positive (tp). If change is detected, and there is no change point within the AT previous steps, consider it a false positive (fp). If no change is detected within Ar steps of change point, we consider it false negative (fn). We count at most one true positive for each actual change point. With these definitions, the precision Prec = tp/(tp + fp), the recall is Rec = tp/(tp + fn), and the Fi-score is their harmonic mean F. = 2. (Prec- Rec) / (Prec + Rec). Note that, while some algorithms allow to infer where the data a change happens, including the proposed MMDEW, we only evaluate the time at which they report a change, all tested approaches allow reporting this value.
13

---

Published in Transactions on Machine Learning Research (01/2023)
MMDEW
AdwinK
IBDD
NEWMA
ScanB
WATCH
1.00
10
Mm
107
0.25
0.00
FMINST
Gan
HAR
MNIST
FMNNST
Ga
BAR
MNGST
Figure 6: Average of percentage of changes detected (PCD) and of mean time to detection (MTD). The dashed line indicates the optimum for PCD. For MTD lower values are better.
Figure 5 shows our results. As ÅT is evahuation-specife parameter, vary it relative to the average distance between change points by factor $ O: Given a data set of length N with changes, we set Ar = 3 N/(n + 1). For 3 (Ar is equal to the average number of steps between change points per respective data set), MMDEW achieves a higher Fi-score than all competilors on all data sets except for Gas, where it still obtains competitive result. Throughout, the proposed algorithm obtains a good balance between precision and recall. Other approaches either have very low precision (for example, less than 20%), or an inferior recall and precision, down to few exceptions. With reduced 8, that is, we allow only a shorter detection delay, the performance of all algorithms decreases on average. For 3 1/2, MMDEW achieves the best Fi score also on four data sets, and, for B = 1/4 (the most challenging setting) three of the tested data sets.
We conclude that the proposed method achieves very good results across all these experiments-especiillly when taking into account the fewer hyperparameters compared to the other approaches that we tested.
Percentage of changes detected and detection delay. To obtain a complete picture of the performance of MMDEW, we also report the "percentage of changes detected" (PCD), that is, the ratio of the number of reported changes and the number of actual change points, and its MTD on the data streams derived from roal-world data. In our context, MTD coincides with the expected detection delay.
Figure 0 collects our results. For PCD, results closer to 100% are better. Here, MMDEW is on par with the closest competitors and consistently, that is, across all data sets, detects approxinately correct number of change points. D3, NEWMA, Scan B-statistics, and WATCH detect too many change points in all cases. This behavior also reflected in their comparably large recall in Figure 5
For MTD, lower values are better. Here, the classification-based D3 performs best most of the cases. MMDEW performs bit worse than D3 but better than the other algorithms on most data sets, with the Gas data the major exception. As the experiments in Figure show, lower Ar tends to lead a lower Fi-score of MMDEW. In other words, MMDEW tends to dctect changes with some delay, but it detects them consistently.
6 Conclusions
We introduced novel change detection algorithm, MMDEW, that builds upon two-sample testing with MMD, which is known to yield powerful tests many domains. To facilitate the efficient computation of MMD, we presented a new data structure, which allows to estimate MMD with polylogarithmie runtime and logarithmic memory complexity. Our experiments on standard benchmark data show that MMDEW obtains the best Fi-score on most data sets. At the same time, MMDEW only has two parameters the level of the statistical test and the choice of kernel. This simplifies the proposed algorithm's application in real-world cases.
14

---

Published in Transactions on Machine Learning Research (01/2025)
Acknowledgments
This work was supported by the German Research Foundation (DFG) Research Training Group GRK 2153: Energy Status Data-Infore Informatics Methods for its Collection, Analysis and Exploitation and by the Baden- Württemberg Foundation via the Elite Program for Postdoctoral Researchers.
References
Saba Abbasi and Abdul Haq- Optimal CUSUM and adaptive CUSUM charts with auxiliary information for process mean. Journal of Statistical Computation and Simulation, 89())3337-361, 2019.
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes- Ortiz. A public domain datasct for human activity rocognition using smartphones. In European Symposium Artificial Neural Networks (ESANN), 2013.
Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Sociely, 68:337-404, 1950.
Alain Berlinet and Christine Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer, 2004.
Albert Bifet and Ricard Gavaldà. Learning from time-changing data with adaptive windowing. In SIAM International Conference on Data Mining (SDM), pp. 443 448, 2007.
Karsten Borgwardt, Elisabetta Ghisu, Felipe Llinarcs-López, Leslic O B'Bray, and Bastian Riec. Graph kernels: Stale-of-the-art and future challenges. Foundations and Trends Machine Learning, 13(5-6):531 712, -72
2020.
George Casella and Roger L. Berger. Statistical inference. Wadsworth & Brooks/Cole, 1990.
Xiuyuan Cheng and Yao Xie. Kernel two-sample lests for manifold data. Bernoulli, 30(4):2572 2597, 2024.
Marco Cuturi. Fast global alignment kernels. In International Conference on Machine Learning (ICML), pp. 929-936, 2011.
Marco Cuturi and Jean-Philippe Vert. The context-tree kernel for strings. Neural Networks, 1818:111 2005.
Tamraparni Dasu, Shankar Krishnan, Dongyu Lin, Suresh Venkatasubramaniin, and Kevin Yi. Change (detection) you can believe in: Finding distributional shifts in data streams. In International Symposium on Intelligent Data Analysis (IDA), volume 5772, pp. 21-34, 2009.
Vinícius M. A. de Souza, Antonio Rafael Sabino Parmezan, Farhan Asif Chowdhury, and Abdullah Mueen. Efficient unsupervised drift detector for fast and high-dimensional data streams. Knowledge and Informa- tion Systems, 6396))1497715277, 2021.
Li Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, pp. 141 142, 2012.
Pedro Domingos and Geoff Hulten. A general framework for mining massive data streams. Journal of Computational and Graphicul Statistics, 12(4):945 949, 2003.
Kamil Faber, Roberto Corizzo, Bartlomiej Sniezynski, Michael Baron, and Nathalie Japkowicz. WATCH: Wasserstein change point detection for high-dimensional time series data. In IEEE International Confer- ence on Big Dala, pp. 4450-44 4459,, 2021.
William J. Faithfull, Juan José Rodríguez Diez, and Ludmila I. Kuncheva. Combining univariate approaches for ensemble change detection in multivariate data. Information Fusion, 45:202-251 212202-11,, 2019.
Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, and Bernhard Schölkopf. Kernel measures of conditional dependence. Tn Advances in Neural Information Processing Systems (NewrIPS), pp. 498-496 2008.
15

---

Published in Transactions on Machine Learning Research (01/2025)
João Gama. Knowledge discovery from data streams. CRC Press, 2010.
Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median heuristic. Technical report, 2018. tttp::///rriv.oorgaa///TT...... 07269
Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels: Hardness results and efficient alterna- tives. Computational Learning Theory and Kernel Machines (COLT), 2777:129-1444 2003.
Ömer Gözüaçik, Alican Büyiükçakir, Hamed R. Bonab, and Fazli Can. Unsupervised concept drift detection with discriminative classifier. In International Conference on Information and Knowledge Management (CIKM), pp. 2365-2368, 2019.
Ronald L. Graham, Donald E. Knuth, and Oren Patashnik. Concrete Mathematics. Addison-Wesley 1994.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13:723-773, 2012.
Zaid Harchaoui and Olivier Cappé. Retrospective multiple change-point estimation with kernels. In IEEE/SP Workshop on Statistical Signal Processing, pp. 768-772, 2007.
Nicolas Keriven, Damien Garreau, and Iacopo Poli. NEWMA: new method for scalable model-freo online change-point detection. JEEE Transactions Signal Processing, 68:3515 3528, 2020.
Franz J. Király and Harald Oberhauser. Kernels for sequentially ordered data. Journal of Machine Learning Research, 20:1 20:1-45, 2019.
Alex Krizhevsky; Geolfrey Hinton, el al. Learning multiple layers of features from tiny images. Technical report, 2009. httpp::/www. cs us.uutoontooooooooooooooooooo... waw-krrie/leaminn--eeeeeee
2009--------RRR pd.
Shuang Li, Yao Xie, Hanjun Dai, and Le Song. Scan B-statistic for kernel change-point detection. Sequential Analysis, 38(4):503-54) 5444, 2019.
Gary Lorden. On excess over the boundary. Annals of Mothematical Statistics, 41:520 527, 1970.
Gary Lorden and Moshe Pollak. Nonanticipating estimation applied to sequential analysis and changepoint detection. The Annals of Statistics, 33(3):1422-144) 2005.
Quentin Mérigot. A multiscale approach optimal transport. In Computer Grophics Forum, volume 30, pp.
2011.
Krikamol Muandet, Kenji Fukumizu, Bharath K. Sriperumbudur, and Bernhard Schölkopf. Kernel mean embedding distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1-2): 1-141, 2017.
E. S. Page. Contimuous inspection schemes. Biometrika, 41:100-115, 1954.
Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Newral Information Processing Systems (NeurIPS), pp. 1177-1184, 2007.
Aaditya Ramdas, Sashank Jakkam Reddi, Barnabás Póczas, Aarti Singh, and Larry A. Wasserman. On the decreasing power of kernel and distance based nonparametric hypothesis tests high dimensions. In Conference on Artificial Intelligence (AAAI), pp. 3571 3577, 2015.
Michael Reed and Barry Simon. Methods of modern mathematical physics. I. Functional analysis. Academic Press, 1972.
Gaetano Romano, Idris A Eckley, and Paul Fearnhead. A log-linear non-parametric online changepoint detection algorithm based on functional pruming. IEEE Transactions on Signal Processing, 2023.
Gordon J Ross. Parametric and nonparametric sequential change detection in R: The epm package. Journal of Statistical Software, 66:1-20, 2015.
16

---

Published in Transactions on Machine Learning Research (01/2025)
Gordon J Ross and Niall M Adams. Two nonparametric control charts for detecting arbitrary distribution
changes. Journal of Quality Technology, 44(2):102-115 2012.
David W Scott. Feasibility of multivariate density estimates. Biometrika, 78(1):197-200 1991.
Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance- based and RKHS-based statistics in hypothesis testing. Anmals of Statistics, 11:263--229) 2013.
Walter A. Shewhart. The application of statistics an aid in maintaining quality of manufactured product. Journal of the Americon Statistical Association, 20(152):544-544 1925.
D. Siegmund and F. S. Venkatraman. Using the generalized likelihood ratio statistic for sequential detection of change-point. The Anmals of Statistics, 23(1):255 271, 1995.
Neil James Alexander Sloane. Entry A001788 in The On-Line Encyclopedia of Integer Sequences, 1999a.
Neil James Alexander Sloane. Entry A036289 in The On-Line Encyclopedia of Integer Sequences, 1999b.
https://oois.oor///03399
Alexander Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A Hilbert space embodding for distri-
butions. In Algorilhmic Learning Theory (ALT), volume 4754, pp. 13 31, 2007.
Ross S. Sparks. CUSUM charts for signalling varying location shifts. Journal of Quality Technology, 32(
157-171, 2000.
Bharath Sriperunbudur, Arthur Grelion, Kenji Fukumizu, Bernhard Schölkopf, and Gert Lanckriet. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11:1517-
1561, 2010.
Bharath K. Sriperumbudur and Zoltán Szabó. Optimal rates for random Fourier features. In Advances in Newral Information Processing Systems (NewrIPS), pp. 1144 1152, 2015.
Ingo Steinwart and Andreas Christmann. Support Vector Machines. Springer, 2008.
Zoltán Szabó and Bharath K. Sriperumbudur Characteristic and universal tensor product kernels. Journal of Machine Learning Reseavch, pp. 233:1 233:29, 2017.
Gábor Szikely and Maria Rizzo. Testing for equal distributions in high dimension. InterStat, 5:1249-1272. 2004.
Gábor Székely and Maria Rizzo. A new test for multivariate normality. Journal of Mullivariate Analysis, 93:58-80 80, 2005.
Gerrit J. J. van den Burg and Christopher K. I. Williams. An evaluation of change point detection algorithms. Technical report, 2020. httts://arxiv.. wwtp:///w//.aa....///2200/ http::///aaarr.rrrr//////0......
Alexander Vergara, Shankar Vembu, Tuba Ayhan, Margaret A Ryan, Margie L Homer, and Ramón Huerta. Chemical gas sensor drift compensation using classifier ensembles. Sensors and Actuators B: Chemical, 166:320-329, 2012.
Roman Vershynin. High- dimensiimn probability. Cambridge University Press, 2018.
Haoyun Wang and Yao Xie. Sequential change-point detection: Computation versus statistical performance. Wiley Interdisciplinary Reviews: Computational Statistics, 16(1):e1628, 2024.
Chris Watkins. Dynamic alignment kernels. In Advances in Neural Information Processing Systems
(NeurIPS), pp. 39-50, 1999.
Song Wei and Yao Xic. Online kernel CUSUM for change-point detection. Technical report, 2022. https:
15070
17

---

Published in Transactions on Machine Learning Research (01/2025)
Han Xiao, Kashif Rasul, and Roland Vollgrff. Fashion-MNIST: novel image dataset for benchmarking machine learning algorithms. Technical report, 2017.
Liyan Xie, George V Moustakides, and Yao Xie. Window-linited CUSUM for sequential change detection. IEEE Transactions on Information Theory, 69()):5990-6005, 2023.
Yao Xie and David Siegmund. Sequential multi-sensor change-point detection. The Annals of Statistics, 41 (2):670 692, 2013.
Wojciech Zaremba, Arthur Gretton, and Matthew B. Blaschko. B-test: A non-parametric, low variance kernel two-sample test. In Advances in Newral Information Processing Systems (NeurIPS), pp- 755 763, 2013.
A Proofs
This section contains additional proofs. The proof of Proposition D is in Section A.1 Proposition
proved
in Section A.2
A.1 Proof of Proposition 1
Proposition 1 follows from the more general result that state below. The statement and proof similar n. Note that recover Gretton et al. (0012
to Gretton et al. [2012] Theorem 8) but do not assume Theorem 8) in the case that m n. We prove Proposition D afterwards.
Proposition 4. Let IP, Q, Pm, Qn be defined as in the main terl, assume 0 < k(x, v) < K for all a, X
P = Q, and > 0. Then
P MMD (Pm:.
K
K
21
Se 2R
m
n
to its expected value. Changing a single one of either z; Bj in this function results in changes of at most 2vK/m, and 2 /n, giving
Proof. First, we bound the difference of MMD (Pmm: Qnn
n+m
Σ c =4 4K
We now apply the bounded differences incquality (recalled Theorem to obtain
PMMMD (i.m. Q - E MMD (Pms @. 2m) 21 S (m=))
The last step is to bound the expectation, which yields
EMMD (Pm, Qn =
k(x; 1T) + n² b
1
Σ
k(yi Mj)
7.n k(1r.15)
n.m
k(yss xie)
m
mm
mn
r==1
+===1
f,1=1
1
EX(Y, Y) - (m (m-1))EAXX, (X, Y) m n
EEkXX,X X)
1
1 (n 1)EA))EXY,X,) -2EA)XX)))
m
n
7
Ek(X, + Ek(Y.Y) Ex(XYY) 1 (BM(X,))
1
m
n
m
7
K $
K
E - EX -X -XX)) XXXX)
m
n
m
n
Inserting this into the previous inequality, we obtain the stated result.
18

---

Published in Transactions on Machine Learning Research (01/2025)
63m
Proposition is now corollary of Proposition 4 which follows by setting = C (k(m+n) and solving for to obtain test of level a.
A.2 Proof of Proposition
To find Pxy,, (OEIS) to discover that "xy; follows the sequence 1, 8, 24, 64, 160, for I 0,1,2.... Thus
implementation of MMDEW and the On-Line Encyclopedia of Integer Sequences
nxy; 244, for <>>00
(10)
and "xyg =1 Sloane 1999b
To find nxx,, notice that nxx, only changes when one merges two windows, which happens for windows of the same size - The algorithm adds this 2 -xx terms, see and, for [ 0, 1, 112,.... we obtain
the recurrence relation
if I = 0, if 1 1, if / 1,
XXX, =
4
2 - -IXX-- + 2. -xy!
with nxx_, =00. Now write
πxx, = nxx,_ + 22 2 [
+ --
(11)
where the brackets are equal to one their argument is true and zero otherwise (using Iverson's convention: Graham et al. 1994h. To find a closed-form expression for (11), we define the ordinary generating function A(=) Σ, a12.. Now, we multiply 111 by 2 and sum on 1, to obtain
-8.3 + 22
A(z) (22 - 1)3
after some algebra, so that
Xxx, = [=9] 8z + 1
(2x 1)3
where [z'] is the coefficient of zl in the series expansion of the generating function A(z). To extract coefficients, we first decompase A(z) as
3
2
1 1.
A(2)
+
1-
2z
(1 - 22)
(1 22)3
which allows us to then find the cocfficients as
3
3 2
[=|]
2 (2x
(1+ 112
[et] (1 - (1-22)3 (1+ (1(1)(( +)
(c)
1 1-28 2z
1)2
where Graham et al. omit the last term as corresponds to [="], which we do not need. Now, adding terms gives
1994 Table 335) implics (a), (b) is 10 shifted, and (c) is Sloane 1999a shifted. We
3 3-2' - 11++) 11 + (1 1)(1 + 2)2 2--1(12 - -1+ 4),
concluding the proof.
19

---

Published in Transactions on Machine Learning Research (01/2025)
B External results
To proof Proposition ! we recall McDiarmid's concentration inequality (Vershynin 2018).
Theorem (Bounded differences inequality). Let X (XI,..., Xn) be a random vector with independent components. Let f R + R be measurable function. Assume that the value of f(x) can change by at most q under an arbitrary change of a single coordinate of (cI,...,Cn) .n. E R Then, for any > 0, we have
212
P(f(X) - 2 exp
Σ 1==
C Hyperparameter optimization settings
We collect the hyperparameter choices that we tested in our experiments on real-world classification data (Section 5.2 in Table and refer to the respective original publications for additional information on the parameter settings.
Table 3: Values chosen for the parameter optimization.
Algorithm
Parameters
Parameter values
MMDEW
Q 8, k w. p.
a E 0.0100,,.01,011,,000
ADWINK
8 (0.05, 0.1, 0.2,
& (0.01,0002,0.055,0111
D3
{100, 200, 500), E (0.1, 0.5), (0.7,0.8, d d=
IBDD
m,
E {10, 20, 500, w (20, 100, 200, w 100), (0.0000000 0.05,
NEWMA
w,a
Scan B
B,w,a
B E (2,3), (100,,0,0000 a (0.01,
WATCH
E, |4
e [1,2,3) x x (25, 100), u e {10,20, 50, 100, 1000, 2000),
w (100, 250, 500, 1000)
D Additional experiments
In this section, collect additional experiments on synthetic data. In Appendix D.1 D.1 the cernel-based approaches and compare their respective EDD/MTD. In Appendix D.2 show that MMD-based approach detects changes in the covariance structure of multivariate data, which aggregated univariate approaches cannot detect reliably.
align the ARLs of
D.1 EDD/MTD of kernel-based approaches on synthetic data
The following experiments compare the EDD of the kernel-basee online change detection approaches con- sidered in the main text and the related work for a fixed ARL on toy data, extending the experiments of Wei Xie (2022 Figure 4). We note that all approaches considered this section compute, for each new observation, test statistic and compare the statistic change flagged. In the case of the proposed algorithm, multiple test statisties (one for each possible split) are computed. To allow for comparison, select the maximum MMD value across all splits, that is, for MMD, we consider the stopping rule
threshold. If the statistic exceeds the threshold, a
Ú (m=+t1
Th inf t: MMD
(12)
s=1...d
k=0
with a fixed > 0, instead of as done in Section Similarly, for MMDEW, we consider 12 but with subsampling applied to the X,-s. The experimental setup is as follows.
To achieve a fixed target ARL EH,T T for a given stopping time T, we run Monte Carlo simulations on 150,000 samples from IP (O, Ia) with d 20 and select as the (targe ARL))-qantile of the
20

---

Published in Transactions on Machine Learning Research (01/2025)
MMDEW
OKCUSUM
ScanB
NewMA
2-
0.3N(0,. (.0.0N (0, 1
Q
0.7N 1,) 0.3. o
G Laplace(O,,
1 Uniff- o""
EDDIMID
15-
10-
40-
10-
5-
20-
5-
0-
0
3.0
3.5
4.0
4.5
5.0
3.0
3.5
4.0
4.5
5.0
3.0
3.5
4,0
4.5
5,0
3.0
3.5
4,0
4.5
5.0
10g ARL
1o810 ARL
log:0
ARL
logLo ARL
Figure 7: EDD/MTD of kernel-based change detectors with a pre-change distribution of P = -N(O,,) = 20, and the indicated post-change distribution (a = 2).
collected lest statistics as threshold. For online kernel CUSUM, we set its parameters Bmax = 50 and N = 15, matching the settings of Wei & Xie 2022, Figure 4). Similarly, for Scan B-statistics and NEWMA, WC sct Bo 50: the remaining parameters of NEWMA then follow from the heuristics detailed by the authors [Krrven et al. 2020).
For approximating the EDD of MMDEW and NEWMA for threshold b, we draw 64 and 400 samples from P. respectively, before sampling from Q Online kernel CUSUM and Scan B-statistics each receive 1,000 samples from P upfront, for computing the variance estimate they require and to use as a reference sample. All approaches use the Gaussian kernel with the bandwidth set by the median heuristic (Garreau et al.
2018
Figure 7 collects our results, with each subfigure corresponding to a different post-change distribution, of which we sample and process 500 elements to find the first time the test statistics exceeds the threshold. We consider the mcan result over 100 repetitions. The results show that OKCUSUM and NEWMA perform similarly across all experiments, with Scan B-statistics performing generally worse in three of the cases. MMDEW achieves the lowest EDD throughout. Specifically, on the mixed distribution Q = = + +1- (), I²Ia) (7 = 0.3), the EDD the proposed method is between 1.62 and 1.82. In the more challenging setting of 7 = 0.7, the EDD of MMDEW is between 3.06 and 3.46. Here, OKCUSUM performs second-best, with an EDD of 32.15 for a target ART. of 1,000 and 61.92 for a target ARL of 100,000. On the Laplace and Uniform distributions, the proposed method improves upon the results of OKCUSUM and NEWMA well, albeit by smaller margin.
While these experiments show that the proposed method improves upon the state-of-the-art, we note that the experiments require obtaining samples from F, which is rarely feasible in practice. In this case, we recommend setting the threshold of MMDEW by the McDiarmid- based bound (Proposition as we do in the experiments the main text (Section
D.2 Comparison with univariate approaches
In this section, we compare the test statistics of the proposed MMDEW, MMD, the Cramer-von-Mine change point model (CvM CPM; Ross & Adams 2012 Ross 2015), and the recent non-parametric Focus (Romano et al., 2023) approach on 20-dimensional multivariate normal data with mean and correlation shift, respectively. CvM CPM and Focus handle univariate data only. To run each on multivariate data, we run one instance per dimension and consider the means of their test statistics. For MMD and MMDEW, our settings are the same as detailed in Appendix D.1 We set the pre-change distribution to P = =N(0,,, Ia);; the respective post-change distributions Q are indicated Figure g For CPM, which updates all previous test statistics upon observing a new sample, we report the test statistics computed after processing 500 samples from the pre-change distribution and 100 samples from the respective post-change distribution; for all other approaches, we report the test statistic computed upon observing each sample.
Note that NEWMA uses random Fourier features Rahimi & Recht 2007 approximate the kernel.
21

---

Published in Transactions on Machine Learning Research (01/2025)
N(1..
NiO 1,13)
wymamm
1.25
MMDEW
1m1m WMW
1.00
1.00-
0.75
wwwhmmmmm
1.00
1.00
0.75
CPM
CPM
20
1-
Focus
15-
Focus
20
Number of samples
Number of samples
Figure 8: Maximum values of the respective test-statistics (20 repetitions, d = 20). A change (indicated by dashed line) occurs after 500 samples, from (Oa,,, Ia) to the distribution indicated on top of the columns, respectively. For the univariate approaches (CPM, Focus), we run one instance per dimension and consider the mean.
Our results are Figure change either the distribution mean (l.h.s)) or the correlation (r.h.s.) lead to an increase of the test statistic of MMDEW and MMD, respectively. Hence, these approaches allow detecting such changes. CPM and Focus correctly identify the change in mean, which is reflected in the univariate marginals they consider. CPM correctly identifies the change point, that is, the maximum value of the test statistic is at 500. When regarding the change in the correlation (the marginals the univariate approaches consider do not change), Focus' test statistic does not reflect the change point. Surprisingly, for CPM, the change in the correlation structure leads to change the test-statistic but the change is identified incorrectly, with the maximum of the test statistic occurring after approximately 530 samples. We conclude that using MMD the proposed MMDEW preferable aggregating univariate change detectors when processing multivariat data, when the changes not reflected in the marginals.
22


Reviews:
[{"review_id": "U2kltF7KRR", "reviewer": "Reviewer_t3v5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths. The proposed algorithm achieves the state-of-the-art utility in many real world change-detection datasets. The runtime is also competitive against the other methods.\n\nWeakness. There is little theoretical justification of the proposed algorithm. For example, it would be nice if there were some statistical analysis of the mean time to detection under simple Gaussian data.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The work addresses change detection in data streams using the kernel-based maximum mean discrepancy (MMD) framework, with a primary focus on computational efficiency.\n\nFor two sample sets of sizes $m$ and $n$, the time complexity of a single discrepancy test is $O(mn)$. Naively applying this to a data stream of length $t$ requires $O(t)$ discrepancy tests, resulting in a total time complexity of $O(t^3)$. This high computational cost is undesirable in practical applications, and the high frequency of testing may lead to increased false positive rates. The paper's key contribution is a novel algorithm that leverages an efficient online approximation of MMD, reducing the per-observation time complexity to $O(\\log^2 t)$. Notably, the proposed algorithm also achieves state-of-the-art F1 scores on 4 out of 5 real-world change detection datasets derived from classification tasks.\n\nThe core idea involves segmenting the data stream into exponentially growing windows, with recent observations assigned smaller windows and older observations grouped into larger ones. Relevant statistics are updated online, enabling the computation of MMD test statistics for positions between windows in $O(\\log t)$ time.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The variable $K$ in Equation(2) is undefined. (though I suppose it has the same meaning as the one in Proposition 1?\n2. The superscript $s$ in Equations(3-5) is a bit confusing. It will be nice if the authors can provide explicit formulas that map $x_i^s$ to the element $x_j$ in the original input data stream $x_1, \\cdots, x_t$.", "broader_impact_concerns": "There are no concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MyUwU9rArc", "reviewer": "Reviewer_C2JC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The manuscript was an enjoyable read. The authors for prioritizing efficiency, which is essential for online applications, particularly in high-frequency scenarios. The method proposed here has potential as a valuable contribution to the online change-point detection literature due to its reduced runtime complexity and ability to handle continuous data streams effectively.\n\nThe reviewer suggests that the paper would benefit from improvements in clarity. Certain areas, in particular those concerning the methodology, could be improved, enhancing readability. \n\nAdditionally, it would be beneficial to more clearly address the limitations and shortcomings of the proposed procedure, which are currently underemphasized.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a novel methodology for performing online non-parametric change-point detection using a two-sample test based on the maximum mean discrepancy (MMD). This approach enables detection of changes between the pre- and post-change distributions in streaming data.\n\nTo facilitate efficient, online computation of MMD, the authors introduce an approximation computed over exponentially increasing windows. This allows the method to operate with a logarithmic runtime complexity per operation, specifically:  $O(\\log^2(t))$ per update, thereby maintaining a near-constant update rate. This efficiency makes the proposed method suitable for high-speed data streams.\n\nBoth simulated and real datasets are present to demonstrate the efficacy of the procedure.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The paper would benefit from a more comprehensive review of existing procedures in efficient online change-point detection, particularly covering statistical and parametric approaches. A recent litterature review of those approches can be found at:\n\n   - Wang, H., & Xie, Y. (2024). *Sequential change‐point detection: Computation versus statistical performance*. *Wiley Interdisciplinary Reviews: Computational Statistics*, 16(1), e1628.\n\n\n2. Section 4 could be organized more coherently to enhance readability and ensure that each component is systematically introduced. The following adjustments are recommended:\n\n   a. Notation and Definitions could be improved. The definition of elements such as $B_s$ and associated variables should be clarified to prevent confusion. For example:\n     - Explicitly define each window $B_s$ as a triple containing a vector of observations in $\\mathbb{R}^s$ and two elements in $\\mathbb{R}$ that represent the partial sums and relevant terms.\n     - Given the central role of $B$, have the authors considered presenting the entire procedure in terms of $B$?. Currently, there are many mixing terms that could be confusing, such as $X_i$, $x_i$, $x_i^s$, $XX_i$, and $XY_i$. Using something like $B_{s, X}$ instead of XX_s or $B_{s, W}$ instead of X_s. \n\n     - Modify the notation in Equation (3) to indicate that $s$ is a set notation rather than an exponent by placing it in brackets.\n\n   b. Introduce the concept of neighbouring windows earlier and in greater detail. For instance:\n     - Equation (7) is somewhat unclear regarding the range of \"i.\" Explain how these ranges operate, especially in cases where split indexing $s$ could be more intuitive.\n     - Also, the reviewer is confused. In Figure 2, it seems that, at step 4, the MMD compares elements in $B_0 \\cup B_1$ (i.e., points $x_1$ and $x_3$) with $B_2$ (i.e., points $x_1$ and $x_4$), which raises questions about whether overlapping data point x_1 might reduce power?\n\n   c. The pseudocode in the paper could be expanded to ensure it is executable with minimal ambiguity. Specifically:\n     - Initialize $B_0$ within the pseudocode.\n     - Clarify line 8 for better readability and understanding.\n     - Consider evaluating across all possible splits to maximize the likelihood of identifying the change location more accurately, rather than iterating sequentially?\n\n   d. The visual example in Figure 2 is helpful, as it clarifies the process of window merging. Including an expanded, larger-scale visual representation of the window structure could further illustrate the evolution of windows and help with understanding.\n\n3. **Synthetic Data Study**  \n   The synthetic data study could be used more effectively to highlight both the strengths and potential limitations of the proposed method:\n\n   a. The reviewer is a bit unsure about the setup for Average Run Length (ARL) and Mean Time to Detection (MTD) in the synthetic data analysis. In assessing MTD, it appears that a change occurs only after 512 observations? Since this may not align with real-world streaming contexts where we do not expect to see a change, and changes occur unpredictably, the authors could consider simulating change points later.\n\n   b. the authors could provide an analysis comparing the control and power of the approximated MMD from the proposed procedure with a direct, non-approximate MMD. Though computationally impractical for sequential use, a simple MMD scheme would offer a valuable baseline for assessing the impact of the proposed approximation, particularly on detecting changes appearing at a later time step.\n\n   c. The authors could add a **power comparison** with competitor methods using the synthetic datasets.  The authors could set a consistent alpha level (e.g., $\\alpha = 0.05$) across methods with an identical run length (e.g., 10,000 observations). Then, introduce a change at a specified time point (e.g., 5,000) and compare MTD values across a range of replicates for all methods.\n\n   d. The synthetic experiments could also be expanded to:   \n     - Include simulations with temporal dependency rather than solely i.i.d. data, e.g. a sinusodal process. This would enhance the applicability of the results to real-world settings. \n     - Additionally, incorporate non-parametric methods from the statistical literature, such as:\n       - Wei, S., & Xie, Y. (2022). *Online kernel CUSUM for change-point detection*. *arXiv preprint arXiv:2211.15070*.\n       - Romano, G., Eckley, I. A., & Fearnhead, P. (2023). *A log-linear non-parametric online change-point detection algorithm based on functional pruning*. *IEEE Transactions on Signal Processing*.\n       - Ross, G. J. (2015). *Parametric and nonparametric sequential change detection in R: The cpm package*. *Journal of Statistical Software*, 66, 1-20.", "broader_impact_concerns": "N/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FK43DLBrRM", "reviewer": "Reviewer_yR8x", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Disclaimer:** This reviewer has worked on CPD in the past but is not up to date with the current state of the art or the relevant challenges of contemporary CPD implementations.  Please take this as an informed, rather than expert, review. \n\n**Strengths**\n- The paper is reasonably clear and, up to particular points (see below), successfully delivers the contribution to the reader. There are, however, some formal statements that need to be improved (see below).\n- The reference to the relevant previous work seems to be correct (both on CPD and the background on MMD)\n- The underlying idea behind the proposed method is valuable and seems to be novel, but it can be better presented (see weaknesses)\n- The experiments, in general, seem thorough. Evaluating CPD methods is difficult (due to metrics and datasets), and the authors overcame that challenge by defining meaningful experiments and performance indicators. \n- The performance of the proposed method is not always the best, but it doesn't have to be (in order to be a valuable contribution). \n\n**Weaknesses**\n- The CPD problem definition in Sec 3 can be explained better. In its current form, one could understand that a different model produces each observation (instead of a _range_ of observations). \n- Is there a proof for Prop 1? Is it needed, or would it be redundant given  Gretton et al. (2012, Theorem 8) ?\n- Why does Lemma 2 have a proof and Lemma 1 does not?\n- Example 1 could be complemented with a diagram\n- Proposition 3 is not proven. At this stage, I wonder if these results/claims fall under the Proposition/Lemma category if the don't need to be proven. \n- \"Figure 1\" is not a figure; it's an algorithm", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a change point detection framework based on MMD. Their method implements temporal windows summarising past observations to achieve lower complexity and memory costs that MMD: $\\mathcal{O}(\\log^2 t)$ and $\\mathcal{O}(\\log t)$ respectively. \n\nThe proposed method is validated experimentally, against different benchmarks and datasets including standard classification data. In this setup, the proposal performed comparatively well against the considered benchmarks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see the weaknesses above", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: As explained above, reviewers and I agree that the paper meets both the "claims and evidence" and "audience" criteria for acceptance in TMLR.
Decision: Accept (poster)