Locally private online change point detection

1 Introduction . Online change point detection has been an active statistical research area for decades , originated from the demand of a reliable quality control mechanism under time and resources constraints [ e.g . 34 ] . In recent years , due to the advance of technology , the applications of online change point detection are well beyond quality control and include climatology , speech recognition , imaging processing , among many others . While collecting the data as they are generated , one wishes to detect the underlying distributional changes as soon as the changes occur . As the ability of collecting and storing data improves exponentially , protecting users ’ privacy has become one of the central concerns of data science . In practice , many of the systems that we monitor contain sensitive information . For instance , change point algorithms are used in cyber security to detect attacks , with the ultimate aim often being to protect private information [ 24 , 30 , 1 ] . Other common application areas include public health [ 21 , 17 ] and finance [ 18 ] , in which many of the data involved are highly personal . Given the prevalence of such problems , we address two questions in this paper : • whether we can detect changes without the need for direct access to the sensitive raw data ; • and what the cost of protecting privacy is in terms of the detection delay and accuracy . Traditional anonymisation of data has been shown to be an outdated method of privacy protection , particularly in multivariate settings [ 28 , 26 ] . Even when direct identifiers such as names and locations are removed , data sets often contain enough information for researchers to identify individual subjects , and hence there is a need to quantify and control the release of privileged information . Formal privacy constraints provide us with a rigorous framework in which we may tackle such problems and explore the fundamental limits of private methods . Constraints of this type have been imposed in analyses 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . carried out by Apple [ 31 , 29 ] , Google [ 16 ] and Microsoft [ 10 ] , and provides organisations with a way of demonstrating the General Data Protection Regulation compliance [ 7 ] . The most popular privacy constraint is that of differential privacy [ 14 ] , which assumes the existence of a third party who can be trusted to handle all the raw data . In situations where this assumption can not be made , which are the focus in our work , we strengthen this constraint and require our algorithms to satisfy local differential privacy [ e.g . 20 , 12 ] , which , in particular , insists that the raw data are not accessed by anyone except its original holder . Solutions to a wide range of statistical problems have been given , and much underlying theory has been developed [ e.g . 35 , 12 , 13 , 27 , 3 , 15 , 19 ] . In this paper , we are concerned with a multivariate nonparametric regression online change point detection problem , with privatised data . To be specific , we assume that the original data are { ( Xi , Yi ) } i∈N∗ ⊂ Rd × R , where the regression functions mi ( x ) = E ( Yi|Xi = x ) , x ∈ Rd , i ∈ N∗ , ( 1 ) satisfy that mi+1 6= mi if and only if i = ∆ ∈ N∗ . The goal is to find a stopping time t̂ , which minimises the detection delay ( t̂−∆ ) + , while controlling the false alarm probability P ( t̂ < ∆ ) . With the concern of maintaining privacy , we do not directly have access to the original data { ( Xi , Yi ) } i∈N∗ , but a privatised version . Specifically , the original data are transmitted through an α-locally differentially private ( α-LDP ) channel for some fixed α > 0 . The privatised data are denoted by { ( Wi , Zi ) } i∈N∗ ⊂ Rd × R. In our upper bounds we restrict attention to non-interactive mechanisms [ see e.g . 12 ] , and so a privacy mechanism is given by a sequence { Qi } i∈N∗ of conditional distributions , with the interpretation that ( Wi , Zi ) | ( Xi , Yi ) = ( xi , yi ) ∼ Qi ( ·| ( xi , yi ) ) . For { Qi } i∈N∗ to satisfy the α-LDP constraint we require that sup i∈N∗ sup A sup ( x , y ) , ( x′ , y′ ) ∈Rd×R Qi ( A| ( x , y ) ) Qi ( A| ( x′ , y′ ) ) ≤ eα . ( 2 ) In our lower bounds , however , we allow mechanisms to be sequentially interactive [ e.g . 12 ] , so that a privacy mechanism is given by { Qi } i∈N∗ , with the interpretation that ( Wi , Zi ) | { ( Xi , Yi , Wi−1 , Zi−1 , . . . , W1 , Z1 ) = ( xi , yi , wi−1 , zi−1 , . . . , w1 , z1 ) ∼ Qi ( ·| ( xi , yi , wi−1 , zi−1 , . . . , w1 , z1 ) ) . Here the α-LDP constraint requires that sup i∈N∗ sup A sup ( x , y ) , ( x′ , y′ ) ∈Rd×R sup w1 , z1 , ... , wi−1 , zi−1 Qi ( A| ( x , y , wi−1 , zi−1 , . . . , w1 , z1 ) ) Qi ( A| ( x′ , y′ , wi−1 , zi−1 , . . . , w1 , z1 ) ) ≤ eα . Since our upper and lower bounds match , up to a logarithmic factor , we may conclude that simpler non-interactive procedures result in optimal performance for this problem . We will assume throughout that α ≤ 1 , though this can be relaxed to α ≤ C for any C > 0 . This restricts attention to the strongest constraints , and is often the regime of primary interest [ e.g . 12 ] . 1.1 Summary of contributions and related literature . To the best of our knowledge , this is the first work on a few fronts . Firstly , this is the first paper to consider change point detection under local privacy constraints . Previous work has focused on the central model of differential privacy , where there exists a third party trusted to have access to all of the data . [ 9 ] use established tools from the central model of differential privacy to detect changes in both the offline and online settings . The pre- and post-change distributions are assumed to be known , and a private version of the likelihood ratio statistic is analysed . Further development of these ideas , in particular the extension to detection of multiple changes in the online setting , is given in [ 39 ] . [ 6 ] give differentially private tests of simple hypotheses , shown to be optimal up to constants , which are then applied to the change point detection problem with known pre- and post-change distributions . In a setting in which the distributions are unknown , [ 8 ] develop private versions of the Mann–Whitney test to detect a change in location . The problem has also been studied under different notions of central privacy [ 23 ] . Secondly , this is the first paper to study the fundamental limits in multivariate nonparametric regression change point detection problems . We have derived the minimax rate of the detection delay , allowing the jump size ‖m∆ − m∆+1‖∞ , the variance of the additive noise σ2 and the privacy constraint α to vary with the location of the change point ∆ . There has been a vast body of literature discussing the detection boundary and optimal estimation in the offline change point analysis [ e.g . 33 , 36 ] . Their counterparts in online change point analysis are relatively scarce and existing work includes univariate mean change [ e.g . 37 ] and dynamic networks [ e.g . 38 ] . On a separate note , multivariate nonparametric regression estimation , under privacy constraints , is studied in [ 4 ] , and classification is studied in [ 2 ] In addition , we have also provided the analysis and results based on the univariate mean online change point detection problem , with privatised data . This is , arguably , the simplest privatised , online change point detection problem . The analysis and results we shown in this paper enrich statisticians ’ toolboxes and serve as a benchmark for more complex problems . 2 Methodology . In this section , we describe our private change point detection algorithm , which takes the privatised data as input . The whole algorithm consists of two key ingredients : ( 1 ) the privacy mechanism and ( 2 ) the change point detection method . The privacy mechanism . Throughout this paper , a binned estimator is the core of the analysis . Recall that the raw data at time point i include a d-dimensional feature vector Xi , which we assume is supported within some bounded set X ⊂ Rd , and a univariate response variable Yi . We denote { Ah , j } j=1 , ... , Nh as a set of cubes of volume hd , such that { X ∩Ah , j } j=1 , ... , Nh is a partition of X , and write xh , j for the centre of Ah , j . The data point ( Xi , Yi ) is then randomised by taking Wi , j = 1 { Xi∈Ah , j } + 4α −1 i , j and Zi , j = [ Yi ] M−M1 { Xi∈Ah , j } + 4Mα −1ζi , j , where { i , j , ζi , j } are independent and identically distributed standard Laplace random variables , and where [ Y ] M−M = min ( M , max ( Y , −M ) ) with M > 0 a truncation parameter . It is shown in Proposition 1 in [ 4 ] [ see also 2 ] that this non-interactive mechanism is an α-LDP channel . We emphasise that privacy is guaranteed without any assumptions on the distribution of ( X , Y ) . The change point detection method . Given data { ( Wi , j , Zi , j ) } , as for online change point detection , we propose Algorithm 1 , with the CUSUM estimator defined in Definition 1 and the nonparametric estimators involved defined in Definition 2 . Algorithm 1 Online change point detection via CUSUM statistics INPUT : { ( Wu , j , Zu , j ) } u , j=1,2 , ... ⊂ Rd × R , { bs , t , 1 ≤ s < t < ∞ , . . . } ⊂ R. t← 1 , FLAG← 0 ; while FLAG = 0 do t← t+ 1 ; FLAG = 1− ∏t−1 s=1 1 { D̂s , t ≤ bs , t } ; end while OUTPUT : t. Definition 1 . Given a sequence { ( Wt , j , Zt , j ) } and a pair of integers 1 ≤ s < t , we define the CUSUM statistic D̂s , t = Nh max j=1 √ s ( t− s ) /t ∣∣m̂1 : s ( xh , j ) − m̂ ( s+1 ) : t ( xh , j ) ∣∣ , where m̂· : · ( · ) is defined in Definition 2 . Definition 2 . Given a sequence { ( Wt , j , Zt , j ) } and a pair of integers 1 ≤ s < t , with h > 0 being our bandwidth parameter , we define the regression function estimator as m̂s : t ( x ) = ν̂s : t ( Ah , j ) µ̂s : t ( Ah , j ) 1 { µ̂s : t ( Ah , j ) ≥ log ( t−s+2 ) t−s+1 } , if x ∈ Ah , j , where ν̂s : t ( Ah , j ) = ( t− s+ 1 ) −1 ∑ s≤i≤t Zi , j and µ̂s : t ( Ah , j ) = ( t− s+ 1 ) −1 ∑ s≤i≤t Wi , j . Algorithm 1 is a standard online change point detection procedure . Whenever a new data point stamped at t is collected , one checks if a change point has occurred by checking if any of ( D̂s , t ) s=1 , ... , t−1 have exceeded pre-specified thresholds . Apparently , both the storage and computation costs of this procedure are high . As pointed out in [ 37 ] , without knowing the pre- and/or post-change point distributions , there is no existing algorithm which can have a constant computational cost O ( 1 ) when a new data point is collected . Of course , one can also be smarter : instead of scanning through s ∈ { 1 , . . . , t− 1 } , one can just scan through a dyadic grid as described in [ 38 ] . The CUSUM statistic D̂s , t , defined in Definition 1 , is a normalised difference between two estimated functions , before and after time point s , evaluated at each cell in the partition . It is used to examine , given all the data up to t , if s is a possible change point . It can be viewed as a sample version of the quantity t−1 max s=1 √ s ( t− s ) t ∥∥∥∥∥s−1 s∑ i=1 mi − ( t− s ) −1 t∑ i=s+1 mi ∥∥∥∥∥ ∞ , where ‖ · ‖∞ is the sup-norm of a function . In [ 25 ] , a similar CUSUM was proposed in studying the nonparametric density change point detection problem . In our setting , we are interested in change points in the regression functions . The construction of estimators is detailed in Definition 2 . This estimator has previously studied in the one sample estimation scenario in [ 4 ] , where it was shown to be a universally strongly consistent estimator of the true regression function .

Summary: The paper studies change point detection for regression under the constraint of local differential privacy (LDP). Its main contribution is a non-interactive (in the sense that all randomizers are chosen ahead of time) algorithm and upper bound on the error of the identified change point, with a sequentially interactive lower bound matching up to logarithmic factors in the failure probability. The paper also provides a non-private benchmark, which is apparently also original.