Locally private online change point detection

1 Introduction . Online change point detection has been an active statistical research area for decades , originated from the demand of a reliable quality control mechanism under time and resources constraints [ e.g . 34 ] . In recent years , due to the advance of technology , the applications of online change point detection are well beyond quality control and include climatology , speech recognition , imaging processing , among many others . While collecting the data as they are generated , one wishes to detect the underlying distributional changes as soon as the changes occur . As the ability of collecting and storing data improves exponentially , protecting users ’ privacy has become one of the central concerns of data science . In practice , many of the systems that we monitor contain sensitive information . For instance , change point algorithms are used in cyber security to detect attacks , with the ultimate aim often being to protect private information [ 24 , 30 , 1 ] . Other common application areas include public health [ 21 , 17 ] and finance [ 18 ] , in which many of the data involved are highly personal . Given the prevalence of such problems , we address two questions in this paper : • whether we can detect changes without the need for direct access to the sensitive raw data ; • and what the cost of protecting privacy is in terms of the detection delay and accuracy . Traditional anonymisation of data has been shown to be an outdated method of privacy protection , particularly in multivariate settings [ 28 , 26 ] . Even when direct identifiers such as names and locations are removed , data sets often contain enough information for researchers to identify individual subjects , and hence there is a need to quantify and control the release of privileged information . Formal privacy constraints provide us with a rigorous framework in which we may tackle such problems and explore the fundamental limits of private methods . Constraints of this type have been imposed in analyses 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . carried out by Apple [ 31 , 29 ] , Google [ 16 ] and Microsoft [ 10 ] , and provides organisations with a way of demonstrating the General Data Protection Regulation compliance [ 7 ] . The most popular privacy constraint is that of differential privacy [ 14 ] , which assumes the existence of a third party who can be trusted to handle all the raw data . In situations where this assumption can not be made , which are the focus in our work , we strengthen this constraint and require our algorithms to satisfy local differential privacy [ e.g . 20 , 12 ] , which , in particular , insists that the raw data are not accessed by anyone except its original holder . Solutions to a wide range of statistical problems have been given , and much underlying theory has been developed [ e.g . 35 , 12 , 13 , 27 , 3 , 15 , 19 ] . In this paper , we are concerned with a multivariate nonparametric regression online change point detection problem , with privatised data . To be specific , we assume that the original data are { ( Xi , Yi ) } i∈N∗ ⊂ Rd × R , where the regression functions mi ( x ) = E ( Yi|Xi = x ) , x ∈ Rd , i ∈ N∗ , ( 1 ) satisfy that mi+1 6= mi if and only if i = ∆ ∈ N∗ . The goal is to find a stopping time t̂ , which minimises the detection delay ( t̂−∆ ) + , while controlling the false alarm probability P ( t̂ < ∆ ) . With the concern of maintaining privacy , we do not directly have access to the original data { ( Xi , Yi ) } i∈N∗ , but a privatised version . Specifically , the original data are transmitted through an α-locally differentially private ( α-LDP ) channel for some fixed α > 0 . The privatised data are denoted by { ( Wi , Zi ) } i∈N∗ ⊂ Rd × R. In our upper bounds we restrict attention to non-interactive mechanisms [ see e.g . 12 ] , and so a privacy mechanism is given by a sequence { Qi } i∈N∗ of conditional distributions , with the interpretation that ( Wi , Zi ) | ( Xi , Yi ) = ( xi , yi ) ∼ Qi ( ·| ( xi , yi ) ) . For { Qi } i∈N∗ to satisfy the α-LDP constraint we require that sup i∈N∗ sup A sup ( x , y ) , ( x′ , y′ ) ∈Rd×R Qi ( A| ( x , y ) ) Qi ( A| ( x′ , y′ ) ) ≤ eα . ( 2 ) In our lower bounds , however , we allow mechanisms to be sequentially interactive [ e.g . 12 ] , so that a privacy mechanism is given by { Qi } i∈N∗ , with the interpretation that ( Wi , Zi ) | { ( Xi , Yi , Wi−1 , Zi−1 , . . . , W1 , Z1 ) = ( xi , yi , wi−1 , zi−1 , . . . , w1 , z1 ) ∼ Qi ( ·| ( xi , yi , wi−1 , zi−1 , . . . , w1 , z1 ) ) . Here the α-LDP constraint requires that sup i∈N∗ sup A sup ( x , y ) , ( x′ , y′ ) ∈Rd×R sup w1 , z1 , ... , wi−1 , zi−1 Qi ( A| ( x , y , wi−1 , zi−1 , . . . , w1 , z1 ) ) Qi ( A| ( x′ , y′ , wi−1 , zi−1 , . . . , w1 , z1 ) ) ≤ eα . Since our upper and lower bounds match , up to a logarithmic factor , we may conclude that simpler non-interactive procedures result in optimal performance for this problem . We will assume throughout that α ≤ 1 , though this can be relaxed to α ≤ C for any C > 0 . This restricts attention to the strongest constraints , and is often the regime of primary interest [ e.g . 12 ] . 1.1 Summary of contributions and related literature . To the best of our knowledge , this is the first work on a few fronts . Firstly , this is the first paper to consider change point detection under local privacy constraints . Previous work has focused on the central model of differential privacy , where there exists a third party trusted to have access to all of the data . [ 9 ] use established tools from the central model of differential privacy to detect changes in both the offline and online settings . The pre- and post-change distributions are assumed to be known , and a private version of the likelihood ratio statistic is analysed . Further development of these ideas , in particular the extension to detection of multiple changes in the online setting , is given in [ 39 ] . [ 6 ] give differentially private tests of simple hypotheses , shown to be optimal up to constants , which are then applied to the change point detection problem with known pre- and post-change distributions . In a setting in which the distributions are unknown , [ 8 ] develop private versions of the Mann–Whitney test to detect a change in location . The problem has also been studied under different notions of central privacy [ 23 ] . Secondly , this is the first paper to study the fundamental limits in multivariate nonparametric regression change point detection problems . We have derived the minimax rate of the detection delay , allowing the jump size ‖m∆ − m∆+1‖∞ , the variance of the additive noise σ2 and the privacy constraint α to vary with the location of the change point ∆ . There has been a vast body of literature discussing the detection boundary and optimal estimation in the offline change point analysis [ e.g . 33 , 36 ] . Their counterparts in online change point analysis are relatively scarce and existing work includes univariate mean change [ e.g . 37 ] and dynamic networks [ e.g . 38 ] . On a separate note , multivariate nonparametric regression estimation , under privacy constraints , is studied in [ 4 ] , and classification is studied in [ 2 ] In addition , we have also provided the analysis and results based on the univariate mean online change point detection problem , with privatised data . This is , arguably , the simplest privatised , online change point detection problem . The analysis and results we shown in this paper enrich statisticians ’ toolboxes and serve as a benchmark for more complex problems . 2 Methodology . In this section , we describe our private change point detection algorithm , which takes the privatised data as input . The whole algorithm consists of two key ingredients : ( 1 ) the privacy mechanism and ( 2 ) the change point detection method . The privacy mechanism . Throughout this paper , a binned estimator is the core of the analysis . Recall that the raw data at time point i include a d-dimensional feature vector Xi , which we assume is supported within some bounded set X ⊂ Rd , and a univariate response variable Yi . We denote { Ah , j } j=1 , ... , Nh as a set of cubes of volume hd , such that { X ∩Ah , j } j=1 , ... , Nh is a partition of X , and write xh , j for the centre of Ah , j . The data point ( Xi , Yi ) is then randomised by taking Wi , j = 1 { Xi∈Ah , j } + 4α −1 i , j and Zi , j = [ Yi ] M−M1 { Xi∈Ah , j } + 4Mα −1ζi , j , where { i , j , ζi , j } are independent and identically distributed standard Laplace random variables , and where [ Y ] M−M = min ( M , max ( Y , −M ) ) with M > 0 a truncation parameter . It is shown in Proposition 1 in [ 4 ] [ see also 2 ] that this non-interactive mechanism is an α-LDP channel . We emphasise that privacy is guaranteed without any assumptions on the distribution of ( X , Y ) . The change point detection method . Given data { ( Wi , j , Zi , j ) } , as for online change point detection , we propose Algorithm 1 , with the CUSUM estimator defined in Definition 1 and the nonparametric estimators involved defined in Definition 2 . Algorithm 1 Online change point detection via CUSUM statistics INPUT : { ( Wu , j , Zu , j ) } u , j=1,2 , ... ⊂ Rd × R , { bs , t , 1 ≤ s < t < ∞ , . . . } ⊂ R. t← 1 , FLAG← 0 ; while FLAG = 0 do t← t+ 1 ; FLAG = 1− ∏t−1 s=1 1 { D̂s , t ≤ bs , t } ; end while OUTPUT : t. Definition 1 . Given a sequence { ( Wt , j , Zt , j ) } and a pair of integers 1 ≤ s < t , we define the CUSUM statistic D̂s , t = Nh max j=1 √ s ( t− s ) /t ∣∣m̂1 : s ( xh , j ) − m̂ ( s+1 ) : t ( xh , j ) ∣∣ , where m̂· : · ( · ) is defined in Definition 2 . Definition 2 . Given a sequence { ( Wt , j , Zt , j ) } and a pair of integers 1 ≤ s < t , with h > 0 being our bandwidth parameter , we define the regression function estimator as m̂s : t ( x ) = ν̂s : t ( Ah , j ) µ̂s : t ( Ah , j ) 1 { µ̂s : t ( Ah , j ) ≥ log ( t−s+2 ) t−s+1 } , if x ∈ Ah , j , where ν̂s : t ( Ah , j ) = ( t− s+ 1 ) −1 ∑ s≤i≤t Zi , j and µ̂s : t ( Ah , j ) = ( t− s+ 1 ) −1 ∑ s≤i≤t Wi , j . Algorithm 1 is a standard online change point detection procedure . Whenever a new data point stamped at t is collected , one checks if a change point has occurred by checking if any of ( D̂s , t ) s=1 , ... , t−1 have exceeded pre-specified thresholds . Apparently , both the storage and computation costs of this procedure are high . As pointed out in [ 37 ] , without knowing the pre- and/or post-change point distributions , there is no existing algorithm which can have a constant computational cost O ( 1 ) when a new data point is collected . Of course , one can also be smarter : instead of scanning through s ∈ { 1 , . . . , t− 1 } , one can just scan through a dyadic grid as described in [ 38 ] . The CUSUM statistic D̂s , t , defined in Definition 1 , is a normalised difference between two estimated functions , before and after time point s , evaluated at each cell in the partition . It is used to examine , given all the data up to t , if s is a possible change point . It can be viewed as a sample version of the quantity t−1 max s=1 √ s ( t− s ) t ∥∥∥∥∥s−1 s∑ i=1 mi − ( t− s ) −1 t∑ i=s+1 mi ∥∥∥∥∥ ∞ , where ‖ · ‖∞ is the sup-norm of a function . In [ 25 ] , a similar CUSUM was proposed in studying the nonparametric density change point detection problem . In our setting , we are interested in change points in the regression functions . The construction of estimators is detailed in Definition 2 . This estimator has previously studied in the one sample estimation scenario in [ 4 ] , where it was shown to be a universally strongly consistent estimator of the true regression function .

Summary: This paper studies the change point detection problems in the multivariate nonparametric regression under the constraint of local differential privacy. Specifically, the goal is to detect changes in the regression function as soon as the change occurs. To achieve LDP, the proposed method first uses a locally private mechanism to transform the data point (x,y) to another random pair and then uses a classical change point detection algorithm with newly defined CUSUM statistics to detect the change. Theoretical false alarm probability analysis and the minimax detection delay properties are given. To quantify the cost of privacy, the paper also provides the optimal rate in the non-private setting.