Network change point localisation under local differential privacy
Abstract
Network data are ubiquitous in our daily life, containing rich but often sensitive information.
In this paper, we expand the current static analysis of privatised networks to a dynamic framework by considering a sequence of networks with potential change points.
We investigate the fundamental limits in consistently localising change points under both node and edge privacy constraints, demonstrating interesting phase transition in terms of the signal-to-noise ratio condition, accompanied by polynomial-time algorithms.
The private signal-to-noise ratio conditions quantify the costs of the privacy for change point localisation problems and exhibit a different scaling in the sparsity parameter compared to the non-private counterparts.
Our algorithms are shown to be optimal under the edge LDP constraint up to log factors.
Under node LDP constraint, a gap exists between our upper bound and lower bound and we leave it as an interesting open problem, echoing the challenges in high-dimensional statistical inference under LDP constraints.
Introduction
Numerous application areas and everyday life routinely generate network data, which contain valuable but often sensitive information [e.g. 33, 31].
Understanding the underlying patterns of network data while preserving individuals’ privacy is crucial in modern data analysis.
Several attempts have been made, but mostly focus on studying single snapshots of networks (a.k a. static networks) and/or subject to central differential privacy constraint, where a central data curator is allowed to handle raw information from all individuals [e.g. 21, 22, 23, 9, 30].
In this paper, we are instead concerned with understanding the dynamics of a sequence of networks (a.k.a.
dynamic networks), under local privacy constraints (LDP), where no one is allowed to handle the raw data of other individuals [e g.
Dynamic networks are usually in the form of a sequence of static networks, along a linear ordering, say time.
In the dynamic networks studies, it is vital to capture the ever changing nature.
A handy and useful way to model the changes is to assume that there exists a sequence of unknown time points, where the underlying distributions change abruptly [e.g. 37, 38].
These unknown time points are referred to as change points.
Identifying change points helps to pinpoint important events, and more accurately estimate underlying distributions, which can be regarded as stationary between two consecutive change points.
Dynamic networks change point analysis has demonstrated its success in climatology [e.g. 28], crime science[e.g. 6] and neuroscience [e.g. 10, 29], to name but a few.
Despite the growing popularity in studying dynamic networks, we have witnessed a vacuum in estimating change points while preserving data owners’ privacy.
Having said this, a line of attack has been made to analyse static network data under LDP constraints, where only data owners have access to their individual raw data [e.g. 34, 40].
The private analysis of network data is complicated by the fact that different LDP conditions are required depending on the information that one wants to protect.
For example, in a relationship network, users may want to protect their edge information, i.e. whether they are connected to someone else or not.
As we argue in Section 2.1, formalisation to protect such information should require minimal trust between users due to the symmetric nature of network data.
In a recommending system network, users may want to protect their entire connection portfolio, representing the purchase status of a user over a collection of products.
In a brain imaging network, a patient may even prefer protecting the entire network from adversarial inference attacks.
In view of the aforementioned state of the art, we list the contributions of this paper below
• To our best knowledge, this is the first time investigating change point localisation in dynamic networks, under LDP constraints.
We consider two dynamic network models, where a sequence of sensitive networks are generated from (a) inhomogeneous Bernoulli networks (IBN) (Definition 1) and (b) bipartite networks with possibly dependent Bernoulli entries (Definition 2).
Multiple change points of the raw network distributions are allowed.
To tailor to the network models, we consider two forms of privacy requirements - edge LDP and node LDP (See Section 2).
• For dynamic IBNs under edge LDP, we show a phase transition in terms of the signal-to-noise ratio, partitioning the whole parameter space into two parts: (1) the infeasibility regime where no algorithm is expected to provide consistent change point estimators and (2) the regime where a computationally-efficient algorithm is shown to output consistent estimators.
The importance of this phase transition is twofold: (1) The transition boundary is different from its counterpart in the non-private case [38], quantifying the cost of preserving edge privacy in localising change points.
(2) We show that a simple randomised response [39] based privacy mechanism is minimax rate-optimal for the purpose of change point localisation. • For bipartite networks under node LDP, we derive an infeasibility regime which is different from that under the edge LDP.
This fundamental difference quantifies the difference between these two different LDP constraints, and can be used to help practitioners designing data collection mechanisms.
We adopt a privacy mechanism proposed in [13], together with a change point estimation routine, providing a consistent change point estimator.
Supported by a minimax lower bound result, our estimator is shown to be minimax rate optimal when the number of columns is of constant order.
When the number of columns is allowed to diverge, a gap between our lower and upper bounds exists.
This echos the well-identified challenges in the high-dimensional privacy research [e.g. 13, 14].
We contribute a high-dimensional network example along with in-depth discussions.
Notation For any matrix A ∈ R M × N , let A ij be the ( i, j ) -th entry of A , A i ∈ R N be the i -th row of
A , A ⊤ be the transpose of A , ∥
A ∥ ∞ = max 1 ≤ i ≤ M, 1 ≤ j ≤ N | A ij | and ∥ A ∥ denote the operator norm of A .
For any matrix B ∈ R M × N , let ( A, B ) =
(cid:80) 1 ≤ i ≤ M, 1 ≤ j ≤ N A ij B ij and ∥ A ∥ F = (cid:112)
( A, A ) be the Frobenius norm of A .
For any vector v ∈ R p , let ∥ v ∥ 1 , ∥ v ∥ 2 , ∥ v ∥ ∞ be the ℓ 1 , ℓ 2 and ℓ ∞ vector norms respectively.
For any set S , let | S | be the cardinality of S .
Let S = S ′ ⊔ S ′′ , if S ′ ∪ S ′′ =
S and
S ′ ∩ S ′′ = ∅ .
Let 1 {·} be the indicator function only taking values in { 0 , 1 } .
For two any functions of T , say f ( T ) and g ( T ) , we write f ( T ) ≳
g ( T ) if there exists constants C > 0 and T {+
+}
such that f ( T ) ≥ Cg ( T ) for any T ≥ T 0 , and write f ( T )
≍ g ( T ) if f ( T ) ≳
g ( T ) and f ( T ) ≲ g ( T ) .
1.1 Problem setup
We consider two parallel models of dynamic networks.
The first one is built upon an IBN model, which covers a wide range of models for undirected networks, including the Erd˝os–Rényi random graph [16], the stochastic block model [18] and the random dot product graph [e.g. 3], among others.
Definition 1 (Inhomogeneous Bernoulli network, IBN) .
A network with node set { 1 , . . . , n } is an inhomogeneous Bernoulli network if its adjacency matrix A ∈ R n × n satisfies that A ij =
A ji = 1 { nodes i, j are connected by an edge } and { A ij , i ≤ j } are independent Bernoulli random variables with E ( A ij ) =
Θ ij .
The second model considered is a bipartite IBN with possibly correlated entries within each row of the biadjacency matrix.
See [2] for more discussions on bipartite networks.
Definition 2 (Bipartite IBN) .
A network with node set { 1 , . . . , n 1 + n 2 } =
V 1 ⊔
V 2 , | V 1 | = nand | V 2 | = n 2 , is a bipartite IBN if its biadjacency matrix A ∈ R n 1 × n 2 satisfies the following.
(1) For i ∈ V 1 and j ∈ V 2 , A ij = 1 { nodes i, j are connected } .
For any i 1 , i 2 ∈ V 1 , i 1 ̸
= i 2 , { A i 1 ,j , j = 1 , . . . , n 2 } and { A i 2 ,j , j = 1 , . .
. , n 2 } are independent.
For any i ∈ V 1 , j ∈ V 2 ,
A ij is a Bernoulli random variable with E ( A ij ) =
Θ ij .
Bipartite IBNs are often used in the recommending system, where each i ∈ V 1 represents a user and each j ∈ V 2 represents a product [e.g. 27, 20].
An important difference between Definitionsand 2 is that, in Definition 1 all entries are assumed to be independent, while in Definition 2, entries within the same row are allowed to be arbitrarily dependent.
Dependence in networks are common in practice, for example the control-flow graph considered in [42] where V 1 corresponds to the set of users and each node in V 2 corresponds to a component within some software application and the dependencies therein are due to the causality between nodes.
The change points are defined formally in Assumption 1 where the magnitude of the distributional change is measured by the normalised Frobenius norm.
The choice of Frobenius norm captures both dense and sparse changes in the network structure, see [38].
Assumption 1.
Let { A ( t ) }
Tt =1 ⊂ { 0 , 1 } n 1 × n 2 be an independent sequence of adjacency matrices of
IBNs defined in Definition 1 (in which case n 1 = n 2 = n ) or biadjacency matrices of bipartite IBNs defined in Definition 2, with E { A ( t ) } =
Θ( t ) .
Assume that there exist { η 1 , . . .
, η K } ⊂ { 2 , . . .
, T } , with 1 = η 0 < η 1 < . . .
<
η K ≤ T < η K
+1 =
T + 1 ,
such that Θ( t ) ̸
= Θ( t − 1) , if and only if t ∈ { η 1 , . . .
, η K } .
Let ∆ =
min K +1 k =1 ( η k − η k − 1 ) be the minimal spacing and κ 0 = min
Kk =1 ∥
Θ( η k ) − Θ( η k − 1)
∥ F / ( √ n 1 n 2 ρ ) be the minimal jump size, where ρ = max Tt =1 ∥ Θ( t ) ∥ ∞ denotes the entry-wise sparsity.
For both models, under privacy constraints to be discussed in Section 2, our goal is to construct consistent estimators { (cid:98) η k } (cid:98)
Kk
=1 of { η k } Kk =1 .
To be specific, { (cid:98) η k } (cid:98) Kk =1 is said to be consistent if ∆ − 1 max Kk =1 | (cid:98) η k − η k | → 0 and (cid:98) K
= K holds with probability tending to 1, as the sample size T grows unbounded.
Lastly, we note that in statistical network analysis, when allowing for entry-wise sparsity, it is usually assumed that ρ ≥ log( n ) /n
[e.g. 38] to ensure there are sufficiently many observed edges.
However,
We do not impose lower bounds on ρ in Assumption 1, since to preserve privacy, the expectations of privatised network entries are inflated by a factor of the privacy level α ∈ (0 , 1) .
Let ρ ′ be the sparsity parameter of the privatised networks.
Such inflation automatically ensures that ρ ′ ≥ log( n ) /n , for any ρ ∈ [0 , 1] and n > 1 (See the proof of Theorem 3).
Network local differential privacy
To formalise different network LDP notions, we first recall a general definition of LDP.
A private mechanism is a conditional distribution, which conditional on raw data, outputs privatised data.
For a pre-specified privacy level α ≥ 0 , a random object Z i taking values in Z is a non-interactive 1 α -LDP version of raw data
X i , if for any raw data x and x ′ , any measurable set S ⊂ Z , it holds that
[Equation]
A privacy mechanism is α -LDP if all output Z i ’s are α -LDP.
We focus on the regime α ∈ (0 , 1) , where the effect of privacy is the strongest and is often the regime of primary interest [e.g. 12, 13, 5, 36].
In view of (1), the LDP constraint ensures that each individual i only has access to their own raw data.
To be specific, the privatised Z i only depends on the raw data
X i .
As for network data, to impose
LDP, it is crucial to formalise what a unit of information includes and who are the owners of each unit of information.
In the rest of this section, we consider two cases arising from different application backgrounds.
in
In epidemiological studies on sexually transmitted diseases, network data are formed by edges linking sexual partners [e.g. 35].
A natural choice of information unit is the existence of sexual relationship among subjects.
Due to the sensitivity of such data, one may wish to consider all parties involved to be the owners of a potential link.
Inspired by such applications, we formalise the edge LDP in
Definition 3.
Definition 3 (Edge α -LDP) .
We say that the privacy mechanism Q = { Q ( t ) ij , 1 ≤ i ≤ j ≤ n, 1 ≤ t ≤ T } is edge α -LDP, if for any integer 1 ≤ t ≤ T , any integer pair 1 ≤ i ≤ j ≤ n , any measurable set S ⊂ Z and any x, x ′ ∈ { 0 , 1 } , it holds that
[Equation]
Definition 3 is seemingly stricter than some existing edge LDP notions [e.g. 34, 40], where, instead of (2), it is required that for any i ∈ { 1 , . . . , n } and any x, x ′ ∈ R n with ∥ x − x ′ ∥ 1 =
[Equation]
It is clear that applying any mechanisms that satisfy (2) to each entry of A i ( t ) guarantees (3).
While appropriate for some settings, the edge LDP defined in (3) possesses some caveats [19], listed below.
• The same piece of information is recorded in the adjacency matrix twice, i.e. A ij ( t ) =
A ji ( t ) , and these two records are treated as being potentially different.
The same quantity is thus randomised twice, leading to some inefficiency. • As a more damning issue, mechanisms satisfying (3) may require trust between nodes.
If a node does not follow the protocol correctly, or their data are intercepted, they may reveal information on other nodes in the network.
This is not the case with LDP mechanisms in other settings, where the privacy of an individual is guaranteed regardless of the behaviour of other individuals.
Our definition (2) does not suffer either of these drawbacks since we only privatise the upper triangular part of the adjacency matrix, and to privatise each edge between two nodes, (2) implicitly requires that both parties to agree on their status and the privatised result so that the trust issue can be prevented.
2.2 Node local differential privacy in bipartite inhomogeneous Bernoulli networks
In a Netflix data set, one may model the viewing history by a dynamic bipartite IBN, where each row represents a user, each column represents a movie and each snapshot of network gathers the viewing information within a short time frame.
It is reasonable to consider an information unit to be the viewing history of a user within a time frame, which is a row in a biadjacency matrix.
Inspired by such applications, we formalise the bipartite node LDP in Definition 4.
Definition 4 (Bipartite node α -LDP) .
We say that the privacy mechanism Q = { Q ( t ) i , 1 ≤ i ≤ n 1 , 1 ≤ t ≤ T } is bipartite node α -LDP, if for any integer 1 ≤ t ≤ T , any integer 1 ≤ i ≤ n 1 , any measurable set S ⊂ Z and any x, x ′ ∈ { 0 , 1 } n 2 , it holds that
[Equation]
Different notions of node LDP have been studied in the literature.
Our definition (4) is consistent with [e.g. 34, 40] while some adopt the definition inherited from central DP allowing the neighbouring networks to have different dimension by either inclusion and deletion of one node [23, 11].
Several works consider the same constraint as (4) under the name user-level LDP [e.g. 24, 43] for different learning tasks.
One appealing feature of bipartite graphs when considering node LDP is that the neighbouring data sets x, x ′ can be protected independently for each node in V 1 , whereas in a general graph, node
LDP should account for the intrinsic symmetry of the adjacency matrix when defining neighbouring data sets [19].
Comparing the two LDP definitions we considered in this section, we see that in
Definition 3 level α privacy is imposed to protect one edge, and in Definition 4 level α privacy is imposed to protect n 2 edges.
For the same privacy parameter α , node privacy is a much more stringent constraint than edge privacy [e.g. 34, 40, 19].
Fundamental limits in consistent change point localisation
Recall that our task is to understand how the underlying distributions of dynamic networks change, especially to provide consistent change point estimators defined in Section 1.1, under certain form of LDP constraints.
Without the concern of privacy, dynamic IBN change point localisation is investigated in [38], where a scaling (namely the signal-to-noise ratio) is proposed to partition the whole parameter space into two regimes: a low signal-to-noise ratio regime (infeasibility regime) where no consistent estimator is guaranteed in a minimax sense, and a high signal-to-noise ratio regime where computationally-efficient algorithms are shown to produce consistent estimators.
Recall the model parameters κ 0 the minimal jump size, ρ the entry-wise sparsity of networks, n the network size and ∆ the minimal spacing.
Without the presence of privacy constraints, the infeasibility regime [38] is
[Equation]
which will serve as the benchmark for us to quantify the cost of privacy.
The first model we study is a dynamic IBN model (Definition 1 and Assumption 1), which is identical to the one studied in [38].
Lemma 1 demonstrates an infeasiblity regime of localising change points in such a model under the edge α -LDP defined in Definition 3.
Lemma 1 (Edge α -LDP) .
Let { A ( t ) } T t =1 ⊂ { 0 , 1 } n × n be a sequence of adjacency matrices satisfying
Assumption 1 with K = 1 and let P Tκ 0 , ∆ ,n,ρ denote their joint distribution.
Consider the class of distributions
P = { P Tκ, ∆ ,n,ρ : κ 20 ≤ min { [68 nρ 2 ∆( e α − 1) 2 ] − 1 , 1 / 4 } , ∆ ≤ T/ 3 } .
Let Q edge α denote the set of all non-interactive privacy mechanisms that satisfy the edge α -LDP constraint in Definition 3, for α ∈ (0 , min { 1 , (2 ρ ) − 1 } ) .
We have that
[Equation]
where η ( P ) denotes the change point location specified by distribution P , the first infimum is taken over all possible non-interactive privacy mechanisms, the second infimum is taken over all measurable functions of the privatised data and the supremum is taken over all raw data’s distributions in the class P .
Lemma 1 studies an LDP minimax lower bound in the framework put forward by [13].
It shows that for dynamic IBNs under edge α -LDP, provided κ 20 ρ 2 n ∆( e α − 1) 2 ≍ κ 20 ρ 2 n ∆ α 2 ≲ 1 , the localisation error ∆ − 1 | (cid:98) η − η ( P ) | ≥ 1 / 12 .
This leads to the infeasiblity regime [Equation]
Comparing (5) and (6), any distribution in the regime (5) also falls in the regime (6), implying that imposing edge α -LDP enlarges the infeasibility regime and makes the localisation task harder.
To be specific, the cost of preserving edge LDP comes from two fronts.
• The effective sample size is decreased from ∆ to ∆ α 2 .
LDP’s impact on the effective sample size is commonly observed in the literature over a wide range of problems [e.g. 13, 7, 5, 25].
• A more interesting and problem-specific cost of LDP is reflected by the role of the sparsity parameter ρ , which power is raised to ρ 2 in (6) from ρ in (5).
Despite that networks have been studied under LDP constraints, such result is the first time seen.
Similar effects have been observed in different problems under LDP constraint, including the impacts on dimensionality [e
g. 5] and smoothness levels [e.g. 25].
It is interesting to see that in a high-dimensional sparse network problem, this problem-specific cost of LDP appears on the sparsity parameter.
The second model we consider is a dynamic bipartite IBN model (Definition 2 and Assumption 1), the change point analysis
of which is not seen in the literature, even
without privacy concerns
In
addition to the rows and columns of bipartite IBNs denoting different entities, which is different from well-studied network models, we also allow potentially arbitrary within-row dependence.
Lemmaestablishes an infeasiblity regime of localising change points in such a model under the bipartite node α -LDP defined in Definition 4.
Lemma 2 (Bipartite node α -LDP) .
Let { A ( t ) }
Tt =1 ⊂ { 0 , 1 } n 1 × n 2 be a sequence of biadjacency matrices satisfying Assumption 1 with K = 1 and let P Tκ 0 , ∆ ,n 1 ,n 2 ,ρ denote their joint distribution.
Consider the class of distributions
[Equation]
Let Q node α denote the set of all non-interactive privacy mechanisms that satisfy the bipartite node α -LDP constraint in Definition 4, for α ∈ (0 , min { 1 , (4 ρ ) − 1 } ) .
We have that
[Equation]
where η ( P ) denotes the change point location specified by distribution P .
In an LDP minimax framework, Lemma 2 shows that provided κ 20 ρ 2 n 1 / 2
∆ α 2 ≲ 1 , the localisation error ∆ − 1 | (cid:98) η − η ( P ) | ≥ 1 / 12 .
This leads to the infeasibility regime
[Equation]
To compare (6) and (7), we first let n 1 = n 2 = n in Lemma 2 for convenience.
The infeasibility regime under the node LDP reads as κ 2 0 ρ 2 n 1 / 2 ∆ α 2 ≲ 1 , which compared to (6) implies that the cost of node LDP is higher than the edge LDP.
To further understand the difference between node LDP and edge LDP, we let n = √ n 1 n 2 in Lemma 1.
The infeasibility regime under the edge LDP reads as κ 20 ρ 2 ( n 1 n 2 ) 1 / 2 ∆ α 2 ≲ 1 , which compared to (7) highlights the difference of n 1 / 2 2 , an extra cost of dimensionality.
The extra cost captures the difference between privatising vectors with possibly correlated entries under node LDP and privatising discrete values under edge LDP.
Lastly, we note that both proofs of Lemma 1 and Lemma 2 follow the convex Le Cam method [e.g. 41] where the key step is bounding the chi square distance between a privatised product distribution and mixture of privatised product distributions.
For the node LDP case, the hard distributions exploit the dependence within rows of the biadjancency matrices.
Consistent private network change point algorithms
We have established infeasibility regimes of change point localisation tasks under different network
LDP constraints in Section 3 and have understood how the privacy preservation makes the tasks fundamentally harder.
In this section, we provide polynomial-time private algorithms to obtain consistent change point estimators outside of the infeasibility regimes.
A private algorithm has two key ingredients: (1) a privacy mechanism and (2) an algorithm with privatised data as inputs.
For the two models we consider in this paper, we adopt the same change point localisation algorithm, while using different privacy mechanisms.
The change point localisation algorithm we consider is the network binary segmentation (NBS) algorithm proposed and studied in [38].
It is shown that NBS provides consistent change point estimators without privacy concerns, under minimax optimal conditions.
For completeness, we include NBS in Algorithm 1 and introduce the CUSUM statistic below.
For any form of data { X i } Ti = and any integer triplet 0 ≤ s < t < e ≤ T , the CUSUM statistic is defined as (cid:101) X
( s,e ) ( t ) =
(cid:115)
e − t ( e − s )( t − s ) t (cid:88) i = s
+
X i − (cid:115) t − s ( e − s )
( e − t )
e (cid:88)
i =
t +
X i .
As pointed out in [38], two sequences of independent networks are required as inputs of Algorithmin order to estimate the Frobenius norm of an IBN. In practice, one can split the data to even and odd indices to obtain two sequences of networks. 4.1 Edge α -LDP To privatise a dynamic IBN (Definition 1) under the edge α -LDP, we apply the randomised response mechanism [39] independently to every edge.
The privacy guarantee follows by virtue of the the randomised response mechanism [15].
To be specific, given data { A ( t ) } Tt =1 ⊂ { 0 , 1 } n × n , let
Algorithm
INPUT: { U ( t ) }
Tt =1 , { V ( t ) }
Tt =1 ⊂ R n 1 × n 2 , { ( α m , β m ) } Mm =1 ⊂ [0 , T ] , τ 1 >for m = 1 , . . .
, M do [ s ′ m , e ′ m ] ← [ s, e ] ∩ [ α m , β m ] , ( s m , e m ) ← [ s ′ m +
64 − 1 ( e ′ m − s ′ m ) , e ′ m − 64 − 1 ( e ′ m − s ′ m )] if e m − s m ≥ 1 then b m ← arg max t = s m +1 ,...,e m − 1 ( ˜ U ( s m ,e m ) ( t ) , ˜ V
( s m ,e m ) ( t ))
a m ← ( (cid:101) U
( s m ,e m ) ( b m ) , (cid:101) V ( s m ,e m ) ( b m ))
else a m ← −end if end for m ∗ ← arg max m =1 ,...,M a m if a m ∗ > τ then add b m ∗ to the set of estimated change points
NBS (( s, b m ∗ ) , { ( α m , β m ) } Mm =1 , τ ) NBS (( b m ∗ + 1 , e ) , { ( α m , β m ) }
Mm =1 , τ ) end if
OUTPUT:
The set of estimated change points.
{ U t,i,j , 1 ≤ i ≤ j ≤ n } Tt =1 be independent Unif [0 , 1] random variables that are independent of { A ( t ) }
Tt =1 .
For any t ∈ { 1 , . . .
, T } and any integer pair 1 ≤ i ≤ j ≤ n , let the privatised data be { A ′ ( t ) }
Tt =1 ⊂ { 0 , 1 } n × n with A ′ ij ( t ) = A ′ ji ( t ) =
(cid:26) A ij (
[Equation]
Note that due to the symmetry of the networks, each edge is only privatised once.
Despite that we are dealing with a high-dimensional, sparse dynamic IBN model, with potentially multiple change points,
Theorem 3 below shows that this, arguably simplest privacy mechanism not only provides consistent change point estimators, but also is optimal in terms of the signal-to-noise ratio condition required.
Theorem 3.
Let { A ( t ) }
Tt =1 and { B ( t ) }
Tt =1 be two independent sequences of adjacency matrices satisfying Assumption 1.
For an arbitrarily small ξ > 0 and an absolute constant c 0 > 0 , assume that
[Equation]
Let { (cid:98) η k } (cid:98)
Kk
=1 be the output of the NBS algorithm, with inputs:
• { A ′ ( t ) }
Tt =1 and { B ′ ( t ) }
Tt =1 , privatised version of { A ( t ) }
Tt =1 and { B ( t ) }
Tt =1 obtained through (8) ; • { ( α m , β m ) } Mm =1 , random intervals whose end points are drawn independently and uniformly from { 1 , . . .
, T } such that max Mm =1 ( β m − α m ) ≤ C R ∆ , for some constant C R > 3 / 2 ; and • tuning parameter τ satisfying c 1 n log 3 / 2 ( T ) <
τ < c 2 κ 20 n 2 ρ 2 ∆ α 2 , where c 1 , c 2 > 0 are absolute constants.
It holds with probability at least 1 − exp { log( T/ ∆) − c 3 M ∆ /T } − c 4 T − c 5 that (cid:98) K [Equation]
K and
[Equation]
where c 3 , c 4 , c 5 , c 6 > 0 are absolute constants.
Theorem 3 shows that, provided M ≳ T ∆ − 1 log( T/ ∆) , it holds with probability tending to one,
[Equation]
where the second inequality is due to (9).
Recalling the consistency definition in Section 1.1, (10) implies the consistency of NBS with randomised response privacy mechanism under edge α -LDP.
In view of the condition (9) and the edge LDP infeasibility regime (6), up to a logarithmic factor, we unveil a phase transition with boundary κ 20 ρ 2 n ∆ α 2 ≍ 1 and show that the randomised response mechanism is optimal in the minimax sense.
This is conceptually interesting since, as pointed out in [34], the privatised network obtained by (8) leads to a dense graph even though the original graph may be sparse and therefore does not represent the original graph well.
However, our result shows that this is the best one can do for change point localisation, at least among non-interactive mechanisms.
4.2 Bipartite node α -LDP
To privatise a dynamic bipartite IBN (Definition 2) under the bipartite node α -LDP, we apply the privacy mechanism developed in Duchi et al.
[12, 13] for privatising vectors with bounded ℓ ∞ norm to each row of the biadjacency matrices.
This privacy mechanism has been used in the analysis of mean estimation [e.g. 13], nonparametric density estimation [e.g. 13, 25] and exact support recovery [e.g. 8] problems under LDP.
Given data { A ( t ) } Tt =1 ⊂ { 0 , 1 } n 1 × n 2 , let { U t,i } T,n 1 t =1 ,i =1 be independent Unif [0 , 1] random variables that are independent of { A ( t ) }
Tt =1 and let { ˜ A ij ( t ) } T,n 1 ,n 2 t =1 ,i =1 ,j =1 be random variables satisfying
P
[Equation]
= [Equation] | [Equation]
[Equation]
t
[Equation]
=
[Equation]
1 | A ij ( t ) } = { 1 + A ij ( t ) } / 2 .
Let
B = C n 2 ( e α + 1) /
( e α − 1) with C − 1 n 2 =
[Equation]
(cid:0)
[Equation]
(cid:1)
[Equation]
The privatised data { A ′ ( t ) } Tt =1 ⊂ { 0 , 1 } n 1 × n 2 are obtained by sampling A ′ i ( t ) ∼   Unif (cid:16) z ∈ { B, − B } n 2 | (cid:80) n 2 j =1 z i
˜ A ij ( t ) ≥ 0 (cid:17) , U t,i ≤ e α / (1 + e α ) , Unif (cid:16) [Equation]
Note that ∥
A i ( t ) ∥ ∞
= 1 for any i = 1 , . .
. , n 1 and t = 1 , . .
, T. Applying (26) in [13] with d = nguarantees that
A i ( t ) is an α -private version of A i ( t ) and therefore satisfies the bipartite node α -LDP constraint.
In Theorem 4, we demonstrate that NBS with inputs obtained through (11) is consistent in localising change points under bipartite node α -LDP constraint.
Theorem 4.
Let { A ( t ) }
Tt =1 and { B ( t ) }
Tt =1 be two independent sequences of biadjacency matrices satisfying Assumption 1.
For an arbitrarily small ξ > 0 and an absolute constant c 0 > 0 , assume that
[Equation]
Let { (cid:98) η k } (cid:98)
Kk
=1 be the output of the NBS algorithm with inputs:
• { A ′ ( t ) }
Tt =1 and { B ′ ( t ) }
Tt =1 , privatised version of { A ( t ) }
Tt =1 and { B ( t ) }
Tt =1 obtained through
) ; • { ( α m , β m ) } Mm =1 , random intervals whose end points are drawn independently and uniformly from { 1 , . . .
, T } such that max Mm =1 ( β m − α m ) ≤ C R ∆ , for some constant C R > 3 / 2 ; and • tuning parameter τ satisfying c 1 n 2 α − 2 log 2 ( Tn 1 n 2 ) max {√ n 1 n 2 , n 2 } <
τ < c 2 κ 20 n 1 n 2 ρ 2 ∆ , where c 1 , c 2 > 0 are absolute constants.
It holds with probability at least 1 − exp { log( T/ ∆) − c 3 M ∆ /T } − c 4 T − c 5 that (cid:98) K
= K and
K max
[Equation]
where c 3 , c 4 , c 5 , c 6 > 0 are absolute constants.
Theorem 4 shows that, provided M ≳ T ∆ − 1 log( T/ ∆) , NBS with privatised inputs through channel (11) is consistent.
When n 2 ≍ 1 , the signal-to-noise ratio condition (12) and the infeasibility regime (7) demonstrate a phase transition with boundary κ 20 ρ 2 n 1 / 2 1 ∆ α ≍ 1 , up to a logarithmic factor.
When n 2 is allowed to diverge, a gap between the infeasibility regime (7) and (12) - the regime where our proposed method is deemed to be consistent - emerges.
The larger n 2 is, the larger the gap is.
It is interesting to understand further what happens within the gap and we leave this as an open problem, which echos the challenging problems in high-dimensional statistical inference under LDP.
Despite this open problem, we would like to point out that the condition (12) already exhibits some fundamental difference between change point localisation and mean estimation problems.
To make this clear, we simplify the model by considering only one change point occurs at ∆ and set n 1 = n 2 = n .
Then, for each i ∈ { 1 , . . . , n } , { A i ( t ) } ∆ t =1 can be regarded as ∆ i.i.d random vectors satisfying that A ij ( t ) ∼ Ber( θ ij ) .
Let θ i = { θ ij } nj =1 ∈ [0 , 1] n .
The fundamental limit of estimating θ i is studied in [14].
With ∆ i.i.d samples, their Corollary 3 gives inf privacy
[Equation]
when α ≲ 1 .
Now, write { θ ′ i } n 1 i =1 for the row means of the distribution after the change point and we have ∥ E { A (∆) } − E { A (∆ + 1) }∥
2F = (cid:80) ni
=1 ∥ θ i − θ ′ i ∥ 22 .
Aggregating (13) over i = 1 , . . . , n , it is natural to expect one needs (cid:80) n 1 i =1 ∥ θ
i − θ ′ i ∥ 22 ≳ n 3 / (∆ α 2 ) in order to detect the change point at ∆ .
However, under the same setup, Theorem 4 shows that consistent change point localisation can be achieved under (12), read as (cid:80) n 1 i =1 ∥ θ i − θ ′ i ∥ 22 ≳ n 2 / (∆ α 2 ) , up to logarithmic factors.
This discrepancy between n 3 and n 2 suggests that the gap between our upper bound and lower bound cannot be closed by straightforwardly applying techniques designed for establishing lower bounds in mean estimation problems.
It also illustrates the general wisdom in the change point literature that, one may afford to sacrifice some accuracy in estimating the underlying distributions, if the goal is just to estimate the change points [e.g. 32].
To conclude this section, we would like to present some result of independent interest.
It is studied in the existing literature [Appendix I.3 in 13] that the privatised output from (11) is unbiased, i.e. E {
A ′ i ( t ) }
=
E { A i ( t ) } , while the covariance structure of the privatised output is unknown.
In
Lemma 5, we carefully analyse the covariance matrix of the privatised output and provide an upper bound on its operator norm.
Due to its independent interest, we denote the raw data vector as
V =
( V i ) ∈
R d and denote its privatised output obtained through (11) as Z = ( Z i
) ∈ R d .
Lemma 5.
For any random vector V ∈ R d with ∥ V ∥ ∞ ≤ 1 , we have that Var
[Equation]
Z i
[Equation]
= B
[Equation]
[Equation]
14) and
[Equation]
where where C d,α ∈ [ C 0 , C 1 ] for some absolute constants C 1 > C 0 > 0 .
Letting Σ Z be the covariance matrix of Z , it holds that
[Equation]
where c > 0 is an absolute constant.
Conclusion
In this paper, we studied network change point localisation problems under two forms of LDP constraints.
New signal-to-noise conditions (9) and (12) are derived and by comparing with the non private counterpart, we quantify the cost of privacy as discussed in Sections 3 and 4.
A change in the scaling of sparsity parameter in the private signal to noise conditions reveals a new challenge of learning dynamic networks with possibly sparse and correlated entries.
The results are summarised in the table below, where for clarity we ignored logarithmic factors and consider n 1 = n 2 = n in the bipartite node LDP case.
No privacy [38]
Edge
[Equation]
Node
[Equation] lower bound (7) [Equation] bound
[Equation]
As for limitations, we have only considered non-interactive privacy mechanisms so far, and it would be interesting to further consider interactive ones.
This could take two possible routes by including interactive mechanisms in the lower bound considerations and/or designing interactive mechanism that improves on the current upper bound.
Both directions present some challenges.
From the lower bound perspective, even in our analysis of non-interactive mechanisms, we identify a technical challenge in controlling the χ 2 -divergence between mixtures of private distributions.
Although some techniques have been developed for discrete distributions [e.g. 4, 1], the counterpart for highdimensional discrete distributions is still largely unexplored.
As for the upper bound, different entries in our network model follow different distributions, which is in sharp contrast to the usual i.i.d. case where interactive methods may be helpful.
We therefore expect that allowing interaction within networks cannot improve the signal to noise ratio condition, while interaction across time points requires novel methodology that can handle temporal dependence, account for the decay of privacy and is suitable for the task of change point localisation.
We leave that as our future work.
Acknowledgements and Disclosure of Funding
The authors would like to thank Harry Giles for helpful discussion and suggesting the idea behind the proof of Theorem 3.
TBB acknowledges the support of an Engineering and Physical Sciences
Reseach Council (EPSRC) New Investigator Award EP/W016117/1.
YY acknowledges the support of an EPSRC Standard Grant EP/V013432/1.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See e.g. Section 3 and Section 4.
(b) Did you describe the limitations of your work?
[Yes] See e.g. Section 4.2 and Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] All theoretical results are stated with clear assumptions including Lemma 1, Lemma 2, Theorem 3, Theorem 4 and Lemma 5.
(b) Did you include complete proofs of all theoretical results?
[Yes] All proofs are collected in supplementary material.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Code is available on openreview.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix A in the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We report the median as a substitute.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] The computation is not intensive and therefore we do not see the need to specify the details.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]