Title: Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime
Venue: NeurIPS 2024
Accepted: yes
Keywords: hypothesis testing

federated learning

distributed inference

goodness-of-fit

differential privacy

communication constraint
Abstract: We study distributed goodness-of-fit testing for discrete distribution under bandwidth and differential privacy constraints. Information constraint distributed goodness-of-fit testing is a problem that has received considerable attention recently. The important case of discrete distributions is theoretically well understood in the classical case where all data is available in one "central" location. In a federated setting, however, data is distributed across multiple "locations" (e.g. servers) and cannot readily be shared due to e.g. bandwidth or privacy constraints that each server needs to satisfy. We show how recently derived results for goodness-of-fit testing for the mean of a multivariate Gaussian model extend to the discrete distributions, by leveraging Le Cam's theory of statistical equivalence. In doing so, we derive matching minimax upper- and lower-bounds for the goodness-of-fit testing for discrete distributions under bandwidth or privacy constraints in the regime where number of samples held locally are large.

Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime

Abstract

We study distributed goodness-of-fit testing for discrete distribution under bandwidth and differential privacy constraints. Information constraint distributed goodness-of-fit testing is a problem that has received considerable attention recently. The important case of discrete distributions is theoretically well understood in the classical case where all data is available in one 'central' location. In a federated setting, however, data is distributed across multiple 'locations' (e.g. servers) and cannot readily be shared due to e.g. bandwidth or privacy constraints that each server needs to satisfy. We show how recently derived results for goodness-of-fit testing for the mean of a multivariate Gaussian model extend to the discrete distributions, by leveraging Le Cam's theory of statistical equivalence. In doing so, we derive matching minimax upper- and lower-bounds for the goodness-of-fit testing for discrete distributions under bandwidth or privacy constraints in the regime where number of samples held locally are large.

Keywords : distributed inference, goodness-of-fit testing, differential privacy, communication constraint, federated learning, statistical equivalence.

1 Introduction

Federated learning is a fundamental problem in statistics and machine learning, where data is distributed across multiple locations (e.g. servers) and cannot readily be shared due to e.g. bandwidth or privacy constraints that each server needs to satisfy. The primary goal in these distributed data settings is to perform a single global inference task, such as hypothesis testing, regression, or classification, by aggregating the local information from each server.

Starting a few decades ago, investigations into distributed settings with bandwidth and other information constraints originated in the electrical engineering community, under the names 'decentralized decision theory / the CEO problem' e.g. [90, 12, 92, 18, 58, 87] or 'inference under multiterminal compression' (see [88] for an overview). These were largely motivated by applications where data is by construction observed and processed locally, such as astronomy, meteorology, seismology, surveillance systems, wireless communication, military radar or air traffic control systems.

Modern federated learning often involves data distributed across siloed data centers (e.g., hospitals) or networks of cellphone users, applied in areas such as word prediction, facial and voice recognition, virtual assistants like Siri or Google Assistant, autonomous vehicles, and earthquake prediction [65, 56, 51, 73, 67, 31]. In these settings, bandwidth often becomes a limited or costly resource [57].

Similarly, with advances in electronic record keeping, privacy has become a more and more pressing issue. These issues are prominent in tech industry products [32], including many federated learning

applications mentioned earlier, as well as in scientific fields like medical sciences [60] and social sciences [69].

Methods that preserve privacy have been around in the statistics community for some time, starting in the 1980's [40, 41]. The current leading formal privacy framework is that of differential privacy (DP), as introduced in [42]. DP is a mathematical guarantee, describing whether results or data sets can be considered 'privacy preserving' and hence can be openly published. Whilst many other privacy frameworks exist, this notion of privacy holds a prominent position both theoretically and practically, finding application within industry giants like Google [45], Microsoft [35], Apple [89], as well as governmental entities such as the US Census Bureau [74].

Quantifying the trade-off between privacy and statistical power means that researchers and data analysts can make an appropriate balance between data privacy and meaningful analysis. Similarly, by quantifying the impact of bandwidth constraints, systems can be designed to work as efficiently as possible within such bandwidth constraints.

The performance of distributed inference under bandwidth or differential privacy is well-studied for various estimation problems. For instance, distributed estimation under differential privacy has been studied for the many-normal-means model, discrete distributions and parametric models in [37, 38, 1, 98, 3], and density estimation [75, 59, 21], and nonparametric regression in [23]. Bandwidth constraints have been studied for the many-normal-means and parametric models in e.g. [101, 39, 77, 20, 97, 50, 26, 25], as well as nonparametric models, including Gaussian white noise [102], nonparametric regression [82], density estimation [17, 4], general, abstract settings [99] and online learning [95]. Distributed adaptive estimation methods under bandwidth constraints, where adaptation occurs to the unknown regularity of the functional parameter of interest, were derived in [82, 83, 25]. Testing simple hypotheses under bandwidth constraints has been studied by e.g. [92] and under differential privacy constraints by [29].

In this paper, we consider goodness-of-fit testing for discrete distributions (i.e. the multinomial model) in scenarios where the number of samples received by each server is large. Specifically, we study testing a simple null versus a composite alternative, in the setting where m servers receive n observations each from a distribution on a sample space of cardinality d , where n is large comparatively to m and d . Recently, such multinomial distributed data have found many applications in areas that handle very large samples over (possibly also large) discrete domains. For example, in population genetics [71, 86] and computer science; where it is used for e.g. information retrieval [100, 76], speech and text and classification [55], text mining [24] and large language models [72]. This has sparked recent interest in studying the statistical decision theoretic properties of the multinomial model, see [15] for an overview.

Deriving minimax rates for goodness-of-fit testing of discrete distributions under bandwidth and differential privacy constraints is particularly challenging when each server holds multiple observations. To date, matching rates have been established only when each machine observes a single observation [9, 10, 5] (see also our discussion of related work below). The techniques used to derive lower-bounds in the aforementioned paper heavily rely on the fact that each server contains only one observation, see [9]. Moreover, whilst tight lower-bounds for the multiple observations case exist for the Gaussian model, the functional analytic techniques used to derive these results heavily rely on Gaussianity, see [85] and [22] for the respective bandwidth constraint and DP lower-bounds. Additionally, lower-bound techniques developed for estimation problems generally do not yield tight impossibility results for goodness-of-fit testing problems (see also the discussion in Section B of the appendix).

We derive matching upper- and lower-bounds for goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints in scenarios where the number of samples n held by each of the servers is large in comparison to d and m ; md log d/ √ n = o (1) . This is achieved by leveraging the theory of statistical equivalence, as introduced by Le Cam (see e.g. [62, 81] for an introduction). Leveraging existing results concerning statistical equivalence of multinomial data with a multivariate Gaussian model proven in [30] allow us to show, roughly speaking, that the distributed goodness-of-fit testing problem for discrete distributions is statistically equivalent a distributed goodness-of-fit testing problem for the mean of a multivariate Gaussian model, and hence the minimax rate for the former problem is the same as the minimax rate for the latter problem, which was established in [85] and [22], for bandwidth and differential privacy constraints respectively. Furthermore, we exploit the bandwidth constraint distributed setting in which these two models have

different minimax rates to show that, when n is small compared to d and m , the multinomial model and multivariate Gaussian model are statistically non-equivalent.

The rest of the paper is organized as follows. After a brief section on related work and notation, the article continues with a precise problem formulation in Section 2. Section 2.1 outlines the distributed framework, for both bandwidth and differential privacy constraints. In Section 2.2, we introduce the problem of distributed goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints. In Section 3, the main results concerning the minimax rates are presented. Section 4 briefly outlines the main idea of the proof technique. Section 5 gives further insight into the comparison between discrete distributions and its comparable multivariate Gaussian model. The article ends with a brief discussion of the derived results. In the appendix, the tools of asymptotic equivalence within the distributed framework are presented and the technical proofs are provided.

1.1 Related work

Minimax goodness-of-fit testing knows a rich literature within the statistics and machine learning communities, see [46, 53, 64, 80, 52]. The d -ary discrete distribution uniformity testing problem bares a close relationship with 'classical' nonparametric goodness-of-fit testing in the sense of [14, 79, 33, 96] and other nonparametric testing problems, see Section 1.4 in [54] and references therein.

For distributed goodness-of-fit testing specifically, much less is known. For multivariate Gaussian models under communication or differential privacy constraints, solutions have been established for the case where each server holds multiple observations. Communication constraints have been studied in [8, 84, 85] and differential privacy constraints in [22], with the authors deriving matching minimax upper- and lower-bounds for the goodness-of-fit testing for the mean of a multivariate Gaussian model.

For testing in discrete distributions, only the scenario where each server receives just one observation has been fully characterized in terms of the minimax rate in [9, 10, 5]. See also [28] for an overview. In these aforementioned works, the authors derive minimax rates goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints. See [48, 78, 11, 2, 19] for investigations specifically under local DP (i.e. one observation per server with DP constraint). Nonparametric goodness-of-fit density testing for under local DP is considered in [36, 61], where in [61], the authors consider adaptation as well. For some investigations into the multiple observations per server case, see [34, 47].

For estimation, the bandwidth constraint estimation discrete distributions in the large sample-perserver case has been studied by [3], who derive matching upper and lower-bounds up to logarithmic factors. However, their technique does not extend to the goodness-of-fit testing problem.

1.2 Notation and notions

Throughout this paper, we shall use the following notation. For two positive sequences a k and b k , we use a k ≲ b k to mean that a k ≤ Cb k for some universal positive constant C . We write a k ≍ b k if both a k ≲ b k and b k ≲ a k , and a k ≪ b k if a k /b k = o (1) .

We denote the maximum of a and b by a ∨ b and the minimum by a ∧ b . For k ∈ N , [ k ] represents the set { 1 , . . . , k } . Universal constants c and C may vary between lines. The Euclidean norm of a vector v ∈ R d is denoted by ∥ v ∥ 2 . For a matrix M ∈ R d × d , ∥ M ∥ represents the spectral norm, and Tr ( M ) denotes its trace. I d is the d × d identity matrix.

A non-negative sequence M k is said to be of poly-logarithmic order in non-negative sequences a k , b k , c k if there exists a constant c > 0 such that M k ≲ (log( a k ) log( b k ) log( c k )) c .

Given measurable spaces ( X , X ) and ( Y , Y ) , a Markov kernel K (between ( X , X ) and target ( Y , Y ) ) is a map K ≡ K ( ·|· ) : Y × X → [0 , 1] with the following two properties: The map x ↦→ K ( A | x ) is measurable for all A ∈ Y , and the map A ↦→ K ( A | x ) is a probability measure on Y for every x ∈ X .

If S is a random variable on a probability space ( X , X , P ) , we let P S denote its push-forward measure , i.e. the measure defined by P S ( B ) := P ( S -1 ( B )) . Weshall use E and E S as the expectation

operator corresponding to P and P S . Random variables X,Y,Z form a Markov chain X → Y → Z whenever their joint distribution P ( X,Y,Z ) disintegrates as d P ( X,Y,Z ) = d P X d P Y | X d P Z | Y .

2 Problem formulation

We begin by formally introducing the general framework of distributed inference.

2.1 The distributed framework

Consider a measurable space ( X , X ) with a statistical model P = { P f : f ∈ F} defined on it. In the distributed framework, we consider j = 1 , . . . , m servers, each receiving data X ( j ) drawn from a given distribution P f ∈ P . Each of the servers communicates a transcript Y ( j ) based on the data to a central server, which in turn computes its solution to the testing problem T ( Y ) ∈ { 0 , 1 } based on the aggregated transcripts Y = ( Y (1) , . . . , Y ( m ) ) . We shall use the convention that T ( Y ) = 1 means rejecting the null hypothesis. The transcript generating mechanisms are then given by Markov kernels { K j } j =1 ,...,m , with the Markov kernel (i.e. conditional distribution) of the transcript Y ( j ) given the data X ( j ) and the randomness U shared by the servers denoted by K j ( ·| X ( j ) , U ) . We formalize this in the following definition.

Definition 1. A distributed testing protocol for the model P consists of a triplet { T, { K j } m j =1 , ( U , U , P U ) } , where { K j } m j =1 is a collection of Markov kernels K j : Y ( j ) ×X×U → [0 , 1] defined on a measurable space ( Y ( j ) , Y ( j ) ) , T : ⊗ m j =1 Y ( j ) →{ 0 , 1 } is a measurable map and ( U , U , P U ) is probability space.

The probability space ( U , U , P U ) is used to (possibly) generate a source of randomness (independent of the data) that is shared by the servers. The distributed protocol is said to have no access to shared randomness or to be a local randomness protocol if P U is trivial 1 . In an abuse of notation, we shall often refer to the entire triplet { T, { K j } j =1 ,...,m , ( U , U , P U ) } using just T .

Given a distributed protocol and i.i.d. data from P f we shall use P f to denote the joint distribution of Y = ( Y (1) , . . . , Y ( m ) ) , the data X under P f m and the shared randomness U ∼ P U . Writing x = ( x (1) , . . . , x ( m ) ) ∈ X m , let x ↦→ K ( A | x, u ) denote the Markov kernel ⊗ m j =1 K j ( ·| x ( j ) , u ) (i.e. the product measure). The independence structure of the data yields that P f m K = ⊗ m j =1 P f K j and the push-forward measure of Y can be seen to disintegrate as

where the second equality follows from the independence of U with the data drawn from P f . The above disintegration of the push-forward measure of Y and the product structure of K can be interpreted as ( X,Y,T ( Y )) forming a Markov chain given U , in the sense of the diagram

The diagram indicates the flow of dependencies. The m servers each obtain data X ( j ) from P f , and generate a transcript Y ( j ) based on the data and shared randomness U . The central server then makes a decision T ( Y ) based on the aggregated transcripts Y . For a definition of Markov kernels and Markov chains, see Section 1.2.

Allowing transcript-generating mechanisms to access both shared and local randomness is important for our analysis, as shared randomness has been found to yield strictly better performance in distributed goodness-of-fit testing, see e.g. [9, 10, 5, 8, 84, 85, 22]. Shared randomness protocols can be seen as a subset of common interactive procedures, such as sequential and blackboard protocols

(see e.g. [6]). The aforementioned paper shows that for discrete distribution goodness-of-fit testing in the single observation per server case, sequential and blackboard protocols offer no benefit over shared randomness protocols. Similarly, for mean shift problems in the multivariate Gaussian case, no advantage of sequential protocols over shared randomness protocols is known, except in the case of estimation with unknown variance [ ? ]. Since we study goodness-of-fit testing for discrete distributions in the large-number-of-observations case by comparing with a Gaussian model with known variance, we restrict the setting of the main article to local and shared randomness protocols only. Nevertheless, our theoretical framework is general enough to handle interactive protocols, which we discuss in Section A.2 of the appendix.

Next, we introduce the notion of a bandwidth constraint in the distributed setting.

Definition 2. A distributed protocol is said to satisfy a b -bit bandwidth constraint if its kernels { K j } j =1 ,...,m are defined on measurable spaces ( Y ( j ) , Y ( j ) ) satisfying |Y ( j ) | ≤ 2 b for j = 1 , . . . , m .

Weuse T ( b ) LR and T ( b ) SR to denote the classes of all local randomness and shared randomness distributed testing protocols with communication budget b per machine, respectively.

Lastly, we introduce the notion of differential privacy in the distributed setting. We will be focusing on the notion of differential privacy as put forward by [43, 44]. Differential privacy provides a mathematical framework that guarantees preservation of privacy in a notion akin to cryptographical guarantees. Formally, a differential privacy constraint on a transcript in our setting is formulated as follows.

Definition 3. Let ϵ ≥ 0 , δ ≥ 0 . The transcript Y ( j ) generated from K j , u ∈ U is said to be ( ϵ, δ ) -differentially private if

A distributed testing protocol { T, { K j } m j =1 , ( U , U , P U ) } , is said be a distributed ( ϵ, δ ) -differentially private testing protocol if { K j } j =1 ,...,m satisfies (2) P U -a.s.

Small values of ϵ and δ ensure that, even when the transcript Y ( j ) is publicly available, the sample X ( j ) underlying Y ( j ) is unidentifiable. We stress that this type of differential privacy guarantee concerns the local data X ( j ) in full, even X ( j ) consists of multiple observations. This is often referred local differential privacy , where the privacy guarantee regards each server as essentially pertaining data to 'one indiviual'. For a thorough introduction on differential privacy guarantees, we refer the reader to [42]. We also note that the use of shared randomness does not affect the privacy guarantee provided by the protocol, as the guarantee holds even if the outcome of the shared randomness is known.

Weuse T ( ϵ,δ ) LR and T ( ϵ,δ ) SR to denote the classes of all local- and shared randomness ( ϵ, δ ) -differentially private distributed testing protocols, respectively. We note that the machinery developed in Section A.2 allows consideration of both types of constraints simultaneously. In the main text of the article, we shall focus on the bandwidth constraint and differential privacy constraint separately as minimax rates for the joint constraints are not known for the Gaussian model we use for comparison to the multinomial model in the main article.

2.2 Distributed goodness-of-fit testing

We start by giving a formal description of sampling from a discrete distribution in the distributed setting. Consider a set with cardinality d ; for simplicity, we take ˜ X = { 1 , . . . , d } . Any probability distribution such a set can be characterized by an element of the d -1 -dimensional probability simplex S d , defined as

In our distributed framework, each server j = 1 , . . . , m observes a data ˜ X ( j ) taking values in { 1 , . . . , d } n

That is, each server obtains n i.i.d. draws from a multinomial distribution with parameter q .

The statistical decision problem of interest shall be that of goodness-of-fit or uniformity testing , i.e. distinguishing the hypotheses

where q 0 = ( q 01 , . . . , q 0 d ) = (1 /d, . . . , 1 /d ) ∈ S d and

for some fixed constant R > 0 . The statistical model under consideration shall be denoted by Q = { Q n q : q ∈ F} .

We define the testing risk for a distributed testing protocol T , for the hypotheses (4) (and statistical model Q ) by sum of the type I and worst case type II error over the alternative class;

The minimax testing risk over a class of distributed protocols T is then defined as inf T ∈ T R Q ( T, H ρ ) .

It is clear that, as ρ tends to 0 , the minimax testing risk should increase. We are interested in finding the so called minimax separation rate , or detection boundary, which is a sequence ρ ∗ depending on the model characteristics n, d , m and T such that the minimax testing risk converges to 0 if ρ ≪ ρ ∗ or 1 if ρ ≫ ρ ∗ .

The minimax separation rate captures how the testing problem becomes easier, or more difficult, for different model characteristics. The minimax rate for the hypothesis above case is ρ 2 ≍ √ d mn when T consists of the class of all testing protocols, as was established in [70] and [94].

When T is taken to be one of the bandwidth or privacy constraint classes of tests, i.e. T ( b ) LR and T ( b ) SR T ( ϵ,δ ) LR and T ( ϵ,δ ) SR , it is sensible to expect ρ ∗ to depend on the bandwidth or differential privacy parameters, b and ( ϵ, δ ) , respectively. In the distributed discrete distribution setupj described above with n = 1 , such minimax rates have been derived in [9, 10]. We discuss these results in the next section, contrasting them with the minimax separation rate derived in this paper for the case where md log d/ √ n = o (1) .

3 Minimax rates in the large sample regime

We now turn to the main results of this paper, which concern the minimax rates for goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints in the large sample regime. We shall show that the minimax rates for the distributed multinomial model under bandwidth and differential privacy constraints are the same as the minimax rates for a d -dimensional distributed Gaussian model, as derived in [85] and [22], respectively.

The first theorem establishes the minimax rate for the distributed multinomial model under bandwidth constraints. A proof can be found in Section D of the appendix.

Theorem 1. Consider sequences m ≡ m ν , b ≡ b ν , d ≡ d ν and n ≡ n ν such that md →∞ whilst

Suppose that ρ ≡ ρ ν is a nonnegative sequence satisfying

Then,

When considering the class of only local randomness protocols (i.e. replacing T ( b ) SR with T ( b ) LR in the above display), the minimax separation rate is given by

The theorem above shows that the minimax rate for the distributed multinomial model under bandwidth constraints is given by (6) in the case of access to shared randomness, and (7) in the case of no access to shared randomness. Both rates are the same as those established for a signal detection problem in a d -dimensional distributed Gaussian model, as derived in [85], Theorems 3.1 and 3.2. In Section 4, we shall provide a proof of this result through the notion of statistical equivalence, where we explicitly use that the multinomial model is asymptotically similar to a specific Gaussian model and a corresponding signal detection problem.

The distributed b -bit bandwidth constraint minimax rate for the hypotheses (4) in the multinomial model with n = 1 is established in [9, 10]. Specifically, they find that

Several aspects of this minimax rate are intriguing. First, unlike in the 'large n case' for the same model and hypothesis ((6) and (7)), there is no elbow effect. Secondly, the benefit (i.e. 'efficiency gain') from an increase in bandwidth is exponential, whereas in the large sample scenario of Theorem 4 it is sub-linear. We shall comment on this 'communication super-efficiency' phenomenon further below.

We now turn to the distributed multinomial model under differential privacy constraints. As in the case of the bandwidth constraint uniformity testing problem, we shall show that the minimax rate for the distributed multinomial model under differential privacy constraints is the same as the minimax rate for a d -dimensional distributed Gaussian model, as derived in [22].

The following theorem describes that the above rates are the minimax rates for uniformity testing in the distributed multinomial model under differential privacy constraints, for shared randomness and local randomness only protocols, respectively.

Theorem 2. For any sequences m ≡ m ν , d ≡ d ν and n ≡ n ν such that md →∞ , md log d √ n ν →∞ → 0 , n -1 / 4 ≪ ϵ ≡ ϵ ν ≤ 1 , δ ≡ δ ν ≍ ( md ) -p for some p > 1 . The minimax separation rate in the distributed multinomial model Q for testing the hypotheses (4) using locally ( ϵ, δ ) -differentially private protocols is

in the case of having access to shared randomness. In the case of having only access to local randomness, it is given by

We provide a proof of the theorem in Section D of the appendix. As with the bandwidth constraint case, the minimax separation rates for the distributed multinomial model under differential privacy constraints are derived by comparing the model and hypothesis test to a signal detection problem for the d -dimensional distributed Gaussian model. The rates for the latter problem follow from the proofs of Theorems 4 and 5 in [22], who describe a more general setup which includes signal detection in the d -dimensional distributed Gaussian model as a special case 2 .

Also in the case of privacy, there is a difference between the one observation per server case minimax rate ( n = 1 ) and the multiple observations per server with local differential privacy case. The

minimax rate in the multinomial model for n = 1 is derived in [9, 5];

Comparing this rate to the rate obtained in Theorem 2, we observe phase transitions in the distributed testing problem for multinomial model under local differential privacy constraints which only occurs if the number of observations locally is large compared to the cardinality of the sample space.

4 Deriving the minimax rates through statistical equivalence

The minimax rates for the distributed multinomial model under bandwidth and differential privacy constraints are derived through the notion of statistical equivalence (Le Cam theory), which is a powerful tool for establishing minimax rates in statistical decision theoretic problems. In this section, we shall provide a brief introduction to statistical equivalence, and show how it can be used to derive the minimax rates for the distributed multinomial model under bandwidth and differential privacy constraints. Further details on the statistical equivalence and a detailed proof are deferred Section A of the appendix.

Le Cam theory is a general framework for decision problems. At the core of this theory is the notion of a distance between statistical models, known as Le Cam's deficiency distance. The objective of this distance is to quantify the extent to which a complex statistical model can be approximated by a more simple one. If a model is close to another model in Le Cam's distance, then there is a mapping of solutions to decision theoretic problems from one model to the other. Whenever the risk of the decision problem is bounded, this means that similar performance can be achieved in the two models. Consequently, studying the complex model can be reduced to studying the corresponding simple model. For an extensive introduction to Le Cam theory, see e.g. [62, 81]. For a brief introduction; [63, 66].

Consider a model P = { P f : f ∈ F} (a collection of probability distributions) on a measurable space ( X , X ) (the sample space). For this article, we consider only models with Polish sample spaces and corresponding Borel sigma-algebras and dominated models, meaning that there exists a sigma-finite measure µ such that P f ≪ µ for all f ∈ F . This greatly simplifies the definition of deficiency, given next.

Given another model Q = { Q f : f ∈ F} indexed by the same set F and sample space ( ˜ X , ˜ X ) , we define the deficiency of P with respect to Q as

where the infimum is taken over all Markov kernels C : ˜ X ×X → [0 , 1] and the probability measure P f C : ˜ X → [0 , 1] is understood as P f C ( A ) := ∫ x ∈X C ( A | x ) dP f ( x ) . This is equivalent to the more general notion of deficiency of [27] for dominated models on Polish spaces (see Proposition 9.2 in [68]).

Le Cam's deficiency distance between P and Q is then defined as ∆( P , Q ) = max { d ( P ; Q ) , d ( Q , P ) } . This semi-metric becomes a metric whenever P and Q are identified whenever d ( P ; Q ) + d ( Q , P ) = 0 . Two sequences of experiments P ν and Q ν are called asymptotically equivalent if their difference ∆( P ν , Q ν ) tends to zero as ν approaches infinity. Conversely, such sequences shall be called asymptotically nonequivalent if ∆( P ν , Q ν ) > c as ν →∞ for a fixed constant c > 0 .

In Section A.2, we prove that models that are close in the Le Cam metric (compared to m ) have similar testing risks in the distributed setup. We leverage this result in combination with the fact that the distributed multinomial model is asymptotically equivalent to a d -dimensional distributed Gaussian model, which we describe next.

Consider for q ∈ F and i = 1 , . . . , d the random variables

Let P f ≡ P n f denote the distribution of X ( j ) = ( X ( j ) 1 , . . . , X ( j ) d ) . Let P denote the corresponding experiment. It is shown in [30] that Q is close to P in the Le Cam metric when d is relatively small compared to n . More precisely, it follows from Theorem 1 and Section 7 in [30] that

where C R > 0 is a constant depending only on R . For the testing problem in Gaussian model, with hypotheses (4), the minimax rate can be derived using the results of [85] in case of bandwidth constraints and [22] in case of differential privacy constraints. The key tool from which the minimax rates can then be derived for the multinomial model is the following lemma, which allows comparison of the minimax testing risks for the multinomial and Gaussian models in regimes where the Le Cam distance is small. Its proof is given in Section A.2 of the appendix.

Lemma 1. Suppose m ∆( Q ; P ) ≤ ϱ for ϱ > 0 . Then, it holds that

where T is either T b SR , T b LR , T ( ϵ,δ ) SR or T ( ϵ,δ ) LR .

5 Statistical non-equivalence of discrete and multivariate Gaussian distributions

Theorem 1 describing uniformity testing in the large sample regime and the result derived for n = 1 , as displayed in (8), shows a striking difference terms of the role of the communication budget. Specifically, in the n = 1 regime, an exponential communication efficiency is observed, whereas in the large sample regime, the benefit is only linear. In this section, we shall provide some explanation for this phenomenon, and shall actually leverage this difference to show that the distributed multinomial model and the distributed Gaussian model are asymptotically non-equivalent: Two models are considered asymptotically nonequivalent if their Le Cam distance remains bounded away from zero, even as the amount of data increases in both models.

The multinomial model is equivalent to a model in which one observes N ( j ) = ( N ( j ) 1 , . . . , N ( j ) d ) taking values in { 1 , . . . , n } d , where N ( j ) k ≡ N ( j ) k ( ˜ X ( j ) ) = ∣ ∣ ∣ { i : ˜ X ( j ) i = k }∣ ∣ ∣ . Let Q ′ denote the model generated by the observations N ( j ) . This model is equivalent to Q , meaning ∆( Q , Q ′ ) = 0 . To see this, note that for x = ( x 1 , . . . , x n ) ∈ { 1 , . . . , d } n ,

after which the aforementioned equivalence follows by the Neyman-Fisher factorization criterion, e.g. Lemma 2 in the appendix. When n is large compared to d , one could standardize the count statistics N ( j ) to obtain a statistic that tends towards a d -dimensional Gaussian random vector. When d and m are not too large with respect to n , one can obtain transcripts and corresponding test statistics from these approximately Gaussian vectors, that resemble those one would consider in the Gaussian model, and attain the corresponding minimax rates.

Since the observation N ( j ) takes values in { 1 , . . . , n } d , the full data can be transmitted whenever there are at least d log 2 n -bits are available per server. However, recalling that the observation ˜ X ( j ) takes values in the space { 1 , . . . , d } n , which has cardinality bounded above by 2 n log 2 d , we also obtain that the full data can be transmitted whenever n log 2 d -bits are available. Consequently, whenever

the distributed problem has the same minimax separation rate for the hypothesis in (4) as the unconstrained problem with nm observations; ρ 2 Q ≍ √ d mn . For the Gaussian problem, this is only the case whenever b ≳ d , as can be seen from Theorem 4. This indicates a kind of 'tipping point' occuring whenever n gets small compared to d , where in a bandwidth constraint distributed setting, the testing problem in for the Gaussian model starts to exhibit very different behavior.

Interestingly, this does not imply that the multinomial model is 'easier' from a distributed testing under bandwidth constraints perspective, as there are sub-regimes in which the Gaussian model has a solution whereas the multinomial model does not and vice versa. It indicates that the 'communication complexity' of the sample space matters in the respective decision problems. We can leverage this fact to obtain a lower-bound on the Le Cam distance between the multinomial model and the Gaussian model; which is the content of the next theorem.

Theorem 3. There exists constants C > 0 and c > 0 such that for any n, d ∈ N with

where P is the experiment generated by the observations in (13) , Q is generated according to (3) , both indexed by F as given in (5) .

The proof of the theorem is given in Section D. It leverages that there exist distributed, b -bit bandwidth constraint settings in which the (distributed) multinomial model allows for consistent goodness-of-fit testing, whereas the (distributed) Gaussian model does not. The result then readily follows from the distributed equivalence results derived in Section A.2. The fact that the separation in the respective (distributed) testing risks occurs for a constant number of servers, yields that the two models are asymptotically nonequivalent whenever √ d/ log 2 ( d ) ≥ d/n ≫ 1 . This reasoning crucially exploits the differing minimax rates that occur under the bandwidth constraint, since without such a constraint, the same goodness-of-fit testing problem of (4) would have similar minimax performance for both of the models.

6 Discussion

We have derived minimax separation rates for uniformity testing in the distributed multinomial model under bandwidth and differential privacy constraints, in the large sample regime where md log d/ √ n = o (1) . When contrasted with existing results for large sample regimes, the minimax rates show that the large sample regime is subject to distinctly different phenomena.

The applicability of our results is somewhat constrained by the requirement that md log d/ √ n = o (1) , which limits the range of model characteristics we can consider. Consequently, further work is needed to understand the behavior of the distributed multinomial model in other regimes. The non-equivalence result in Theorem 3 indicates that the distributed multinomial model and the distributed Gaussian model are fundamentally different regarding distributed statistical decision problems when the sample size is small. Therefore, direct analysis of the distributed multinomial model might be necessary, requiring new techniques to derive minimax rates. We note, however, that this pertains to the specific Gaussian model formulated in (13), and there might be a different Gaussian model that is equivalent to the distributed multinomial model even in the small n regime.

The results in this paper are derived through the notion of statistical equivalence, which is a powerful tool for establishing minimax rates in statistical decision theoretic problems. The results and techniques can be applied more generally to other distributed inference problems, and proving more general results concerning statistical equivalence and distributed inference is an interesting avenue for future research.

A downside of leveraging statistical equivalence is that it generally does not provide a direct path to obtain methods that are minimax rate optimal. However, Theorem 1 and Section 7 in [30] provide a specific transformation that converts the local multinomial sample into a statistic approximately distributed as a Gaussian random vector. Such a transformation, combined with the rate optimal methods given in [85] and [22], provide guidance to construct methods that attain the minimax rates described in this article.

Acknowledgments and Disclosure of Funding

The author is grateful to the anonymous reviewers for their valuable feedback and suggestions and to Aad van der Vaart for his thorough reading of an earlier draft.

it holds that

References

ACHARYA, J., BONAWITZ, K., KAIROUZ, P., RAMAGE, D., AND SUN, Z. Context aware local differential privacy. In Proceedings of the 37th International Conference on Machine Learning (13-18 Jul 2020), H. D. III and A. Singh, Eds., vol. 119 of Proceedings of Machine Learning Research , PMLR, pp. 52-62.

ACHARYA, J., CANONNE, C., FREITAG, C., AND TYAGI, H. Test without trust: Optimal locally private distribution testing. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics (16-18 Apr 2019), K. Chaudhuri and M. Sugiyama, Eds., vol. 89 of Proceedings of Machine Learning Research , PMLR, pp. 2067-2076.

ACHARYA, J., CANONNE, C., LIU, Y., SUN, Z., AND TYAGI, H. Distributed estimation with multiple samples per user: Sharp rates and phase transition. In Advances in Neural Information Processing Systems (2021), M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34, Curran Associates, Inc., pp. 18920-18931.

ACHARYA, J., CANONNE, C., SINGH, A. V., AND TYAGI, H. Optimal rates for nonparametric density estimation under communication constraints. Advances in Neural Information Processing Systems 34 (2021), 26754-26766.

ACHARYA, J., CANONNE, C. L., FREITAG, C., SUN, Z., AND TYAGI, H. Inference under information constraints iii: Local privacy constraints. IEEE Journal on Selected Areas in Information Theory 2 , 1 (2021), 253-267.

ACHARYA, J., CANONNE, C. L., LIU, Y., SUN, Z., AND TYAGI, H. Interactive inference under information constraints. IEEE Transactions on Information Theory 68 , 1 (2022), 502516.

ACHARYA, J., CANONNE, C. L., SUN, Z., AND TYAGI, H. Unified lower bounds for interactive high-dimensional estimation under information constraints. Advances in Neural Information Processing Systems 36 (2024).

ACHARYA, J., CANONNE, C. L., AND TYAGI, H. Distributed signal detection under communication constraints. In Conference on Learning Theory (2020), PMLR, pp. 41-63.

ACHARYA, J., CANONNE, C. L., AND TYAGI, H. Inference under information constraints i: Lower bounds from chi-square contraction. IEEE Transactions on Information Theory 66 , 12 (2020), 7835-7855.

ACHARYA, J., CANONNE, C. L., AND TYAGI, H. Inference under information constraints ii: Communication constraints and shared randomness. IEEE Transactions on Information Theory 66 , 12 (2020), 7856-7877.

ACHARYA, J., SUN, Z., AND ZHANG, H. Differentially private testing of identity and closeness of discrete distributions. Advances in Neural Information Processing Systems 31 (2018).

AHLSWEDE, R., AND CSISZÁR, I. Hypothesis testing with communication constraints. IEEE transactions on information theory 32 , 4 (1986), 533-542.

AHN, S., CHEN, W.-N., AND ÖZGÜR, A. Estimating sparse distributions under joint communication and privacy constraints. In 2022 IEEE International Symposium on Information Theory (ISIT) (2022), pp. 3144-3149.

AN, K. Sulla determinazione empirica di una legge didistribuzione. Giorn Dell'inst Ital Degli Att 4 (1933), 89-91.

BALAKRISHNAN, S., AND WASSERMAN, L. Hypothesis testing for high-dimensional multinomials: A selective review. The Annals of Applied Statistics 12 , 2 (2018), 727 - 749.

BARNES, L. P., HAN, Y., AND ÖZGÜR, A. Fisher information for distributed estimation under a blackboard communication protocol. In 2019 IEEE International Symposium on Information Theory (ISIT) (2019), IEEE, pp. 2704-2708.

BARNES, L. P., HAN, Y., AND ÖZGÜR, A. Lower bounds for learning distributions under communication constraints via fisher information. The Journal of Machine Learning Research 21 , 1 (2020), 9583-9612.

BERGER, T., AND ZHANG, Z. On the ceo problem. In Proceedings of 1994 IEEE International Symposium on Information Theory (1994), pp. 201-.

BERRETT, T., AND BUTUCEA, C. Locally private non-asymptotic testing of discrete distributions is faster using interactive mechanisms. Advances in Neural Information Processing Systems 33 (2020), 3164-3173.

BRAVERMAN, M., GARG, A., MA, T., NGUYEN, H. L., AND WOODRUFF, D. P. Communication lower bounds for statistical estimation problems via a distributed data processing inequality. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing (2016), pp. 1011-1020.

BUTUCEA, C., DUBOIS, A., KROLL, M., AND SAUMARD, A. Local differential privacy: Elbow effect in optimal density estimation and adaptation over Besov ellipsoids. Bernoulli 26 , 3 (2020), 1727 - 1764.

CAI, T. T., CHAKRABORTY, A., AND VUURSTEEN, L. Federated nonparametric hypothesis testing with differential privacy constraints: Optimal rates and adaptive tests, 2024. arXiv:2406.06749 [math.ST].

CAI, T. T., CHAKRABORTY, A., AND VUURSTEEN, L. Optimal federated learning for nonparametric regression with heterogeneous distributed differential privacy constraints, 2024. arXiv:2406.06755 [math.ST].

CAI, T. T., KE, Z. T., AND TURNER, P. Testing high-dimensional multinomials with applications to text analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology 86 , 4 (02 2024), 922-942.

CAI, T. T., AND WEI, H. Distributed adaptive gaussian mean estimation with unknown variance: Interactive protocol helps adaptation. The Annals of Statistics 50 , 4 (2022), 19922020.

CAI, T. T., AND WEI, H. Distributed gaussian mean estimation under communication constraints: Optimal rates and communication-efficient algorithms. Journal of Machine Learning Research 25 , 37 (2024), 1-63.

CAM, L. L. Sufficiency and Approximate Sufficiency. The Annals of Mathematical Statistics 35 , 4 (1964), 1419 - 1455.

CANONNE, C. L., ET AL. Topics and techniques in distribution testing: A biased but representative sample. Foundations and Trends® in Communications and Information Theory 19 , 6 (2022), 1032-1198.

CANONNE, C. L., KAMATH, G., MCMILLAN, A., SMITH, A., AND ULLMAN, J. The structure of optimal private tests for simple hypotheses. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing (2019), pp. 310-321.

CARTER, A. V. Deficiency distance between multinomial and multivariate normal experiments. The Annals of Statistics 30 , 3 (2002), 708 - 730.

COCHRAN, E. S., LAWRENCE, J. F., CHRISTENSEN, C., AND JAKKA, R. S. The quakecatcher network: Citizen science expanding seismic horizons. Seismological Research Letters 80 , 1 (2009), 26-30.

CRAIN, M. The limits of transparency: Data brokers and commodification. new media & society 20 , 1 (2018), 88-104.

CRAMÉR, H. On the composition of elementary errors: First paper: Mathematical deductions. Scandinavian Actuarial Journal 1928 , 1 (1928), 13-74.

DIAKONIKOLAS, I., GOULEAKIS, T., KANE, D. M., AND RAO, S. Communication and memory efficient testing of discrete distributions. In Proceedings of the Thirty-Second Conference on Learning Theory (Phoenix, USA, 25-28 Jun 2019), A. Beygelzimer and D. Hsu, Eds., vol. 99 of Proceedings of Machine Learning Research , PMLR, pp. 1070-1106.

DING, B., KULKARNI, J., AND YEKHANIN, S. Collecting telemetry data privately. Advances in Neural Information Processing Systems 30 (2017).

DUBOIS, A., BERRETT, T., AND BUTUCEA, C. Goodness-of-Fit Testing for Hölder Continuous Densities Under Local Differential Privacy. In Foundations of Modern Statistics , vol. PROMS-425 of Springer Proceedings in Mathematics & Statistics . Springer International Publishing, 2023, pp. 53-119.

DUCHI, J. C., JORDAN, M. I., AND WAINWRIGHT, M. J. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science (2013), IEEE, pp. 429-438.

DUCHI, J. C., JORDAN, M. I., AND WAINWRIGHT, M. J. Minimax optimal procedures for locally private estimation. Journal of the American Statistical Association 113 , 521 (2018), 182-201.

DUCHI, J. C., JORDAN, M. I., WAINWRIGHT, M. J., AND ZHANG, Y. Optimality guarantees for distributed statistical estimation. arXiv preprint arXiv:1405.0782 (2014).

DUNCAN, G., AND LAMBERT, D. The risk of disclosure for microdata. Journal of Business &Economic Statistics 7 , 2 (1989), 207-217.

DUNCAN, G. T., AND PEARSON, R. W. Enhancing access to microdata while protecting confidentiality: Prospects for the future. Statistical Science 6 , 3 (1991), 219-232.

DWORK, C. Differential privacy. In International colloquium on automata, languages, and programming (2006), Springer, pp. 1-12.

DWORK, C., MCSHERRY, F., NISSIM, K., AND SMITH, A. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3 (2006), Springer, pp. 265-284.

DWORK, C., AND SMITH, A. Differential privacy for statistics: What we know and what we want to learn. Journal of Privacy and Confidentiality 1 , 2 (2010).

ERLINGSSON, U., PIHUR, V., AND KOROLOVA, A. Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (New York, NY, USA, 2014), CCS '14, Association for Computing Machinery, p. 1054-1067.

ERMAKOV, M. Asymptotically minimax tests for nonparametric hypotheses concerning the distribution density. Journal of Soviet Mathematics 52 (1990), 2891-2898.

FISCHER, O., MEIR, U., AND OSHMAN, R. Distributed uniformity testing. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing (New York, NY, USA, 2018), PODC '18, Association for Computing Machinery, p. 455-464.

GABOARDI, M., LIM, H., ROGERS, R., AND VADHAN, S. Differentially private chi-squared hypothesis testing: Goodness of fit and independence testing. In Proceedings of The 33rd International Conference on Machine Learning (New York, New York, USA, 20-22 Jun 2016), M. F. Balcan and K. Q. Weinberger, Eds., vol. 48 of Proceedings of Machine Learning Research , PMLR, pp. 2111-2120.

HAN, Y., MUKHERJEE, P., OZGUR, A., AND WEISSMAN, T. Distributed statistical estimation of high-dimensional and nonparametric distributions. In 2018 IEEE International Symposium on Information Theory (ISIT) (2018), IEEE, pp. 506-510.

HAN, Y., ÖZGÜR, A., AND WEISSMAN, T. Geometric lower bounds for distributed parameter estimation under communication constraints. In Conference On Learning Theory (2018), PMLR, pp. 3163-3188.

HARD, A., RAO, K., MATHEWS, R., RAMASWAMY, S., BEAUFAYS, F., AUGENSTEIN, S., EICHNER, H., KIDDON, C., AND RAMAGE, D. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 (2018).

INGSTER, Y., AND SUSLINA, I. A. Nonparametric goodness-of-fit testing under Gaussian models , vol. 169. Springer Science & Business Media, 2003.

INGSTER, Y. I. Asymptotically minimax hypothesis testing for nonparametric alternatives. i, ii, iii. Math. Methods Statist 2 , 2 (1993), 85-114.

INGSTER, Y. I., AND SUSLINA, I. A. Nonparametric Goodness-of-Fit Testing Under Gaussian Models , vol. 169 of Lecture Notes in Statistics . Springer New York, New York, NY, 2003.

JURAFSKY, D., AND MARTIN, J. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , vol. 2. 02 2008.

KONE ˇ CN ` Y, J., MCMAHAN, H. B., RAMAGE, D., AND RICHTÁRIK, P. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 (2016).

KONE ˇ CN ` Y, J., MCMAHAN, H. B., YU, F. X., RICHTÁRIK, P., SURESH, A. T., AND BACON, D. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).

KREIDL, O. P., TSITSIKLIS, J. N., AND ZOUMPOULIS, S. I. On Decentralized Detection With Partial Information Sharing Among Sensors. IEEE Transactions on Signal Processing 59 , 4 (Apr. 2011), 1759-1765. Number: 4.

KROLL, M. On density estimation at a fixed point under local differential privacy. Electronic Journal of Statistics 15 , 1 (2021), 1783 - 1813.

KULYNYCH, J., AND GREELY, H. T. Clinical genomics, big data, and electronic medical records: reconciling patient rights with research when privacy and science collide. Journal of Law and the Biosciences 4 , 1 (2017), 94-132.

LAM-WEIL, J., LAURENT, B., AND LOUBES, J.-M. Minimax optimal goodness-of-fit testing for densities and multinomials under a local differential privacy constraint. Bernoulli 28 , 1 (2022), 579-600.

LE CAM, L. Asymptotic methods in statistical decision theory . Springer Science & Business Media, 2012.

LE CAM, L., AND YANG, G. L. Asymptotics in statistics: some basic concepts . Springer Science & Business Media, 2000.

LEPSKII, O. Asymptotically minimax adaptive estimation. i: Upper bounds. optimally adaptive estimates. Theory of Probability & Its Applications 36 , 4 (1992), 682-697.

LI, T., SAHU, A. K., TALWALKAR, A., AND SMITH, V. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine 37 , 3 (2020), 50-60.

MARIUCCI, E. Le cam theory on the comparison of statistical models. arXiv preprint arXiv:1605.03301 (2016).

NGUYEN, A., DO, T., TRAN, M., NGUYEN, B. X., DUONG, C., PHAN, T., TJIPUTRA, E., AND TRAN, Q. D. Deep federated learning for autonomous driving. In 2022 IEEE Intelligent Vehicles Symposium (IV) (2022), IEEE, pp. 1824-1830.

NUSSBAUM, M. Asymptotic equivalence of density estimation and Gaussian white noise. The Annals of Statistics 24 , 6 (1996), 2399 - 2430.

OBERSKI, D. L., AND KREUTER, F. Differential privacy and social science: An urgent puzzle. Harvard Data Science Review 2 , 1 (2020), 1-21.

PANINSKI, L. A Coincidence-Based Test for Uniformity Given Very Sparsely Sampled Discrete Data. IEEE Transactions on Information Theory 54 , 10 (Oct. 2008), 4750-4755.

PRITCHARD, J. K., STEPHENS, M., AND DONNELLY, P. Inference of population structure using multilocus genotype data. Genetics 155 , 2 (2000), 945-959.

RADFORD, A., WU, J., CHILD, R., LUAN, D., AMODEI, D., SUTSKEVER, I., ET AL. Language models are unsupervised multitask learners. OpenAI blog 1 , 8 (2019), 9.

RAMASWAMY, S., MATHEWS, R., RAO, K., AND BEAUFAYS, F. Federated learning for emoji prediction in a mobile keyboard, 2019.

RODRIGUEZ, I. M., SEXTON12, W. N., SINGER, P. E., AND VILHUBER, L. The modernization of statistical disclosure limitation at the us census bureau.

SART, M. Density estimation under local differential privacy and Hellinger loss. Bernoulli 29 , 3 (2023), 2318 - 2341.

SCHÜTZE, H., MANNING, C. D., AND RAGHAVAN, P. Introduction to information retrieval , vol. 39. Cambridge University Press Cambridge, 2008.

SHAMIR, O. Fundamental limits of online and distributed algorithms for statistical learning and estimation. Advances in Neural Information Processing Systems 27 (2014), 163-171.

SHEFFET, O. Locally private hypothesis testing. In International Conference on Machine Learning (2018), PMLR, pp. 4605-4614.

SMIRNOV, N. Table for Estimating the Goodness of Fit of Empirical Distributions. The Annals of Mathematical Statistics 19 , 2 (1948), 279 - 281.

SPOKOINY, V. G. Adaptive hypothesis testing using wavelets. The Annals of Statistics 24 , 6 (Dec. 1996).

STRASSER, H. Mathematical theory of statistics: statistical experiments and asymptotic decision theory . No. 7 in De Gruyter studies in mathematics. W. de Gruyter, Berlin ; New York, 1985.

SZABO, B., AND VAN ZANTEN, H. Adaptive distributed methods under communication constraints. The Annals of Statistics 48 , 4 (2020), 2347-2380.

SZABÓ, B., AND VAN ZANTEN, H. Distributed function estimation: Adaptation using minimal communication. Mathematical Statistics and Learning 5 , 3 (2022), 159-199.

SZABÓ, B., VUURSTEEN, L., AND VAN ZANTEN, H. Optimal distributed composite testing in high-dimensional gaussian models with 1-bit communication. IEEE Transactions on Information Theory 68 , 6 (2022), 4070-4084.

SZABÓ, B., VUURSTEEN, L., AND VAN ZANTEN, H. Optimal high-dimensional and nonparametric distributed testing under communication constraints. The Annals of Statistics 51 , 3 (2023), 909-934.

TAI, Y. C., AND SPEED, T. P. A multivariate empirical Bayes statistic for replicated microarray time course data. The Annals of Statistics 34 , 5 (2006), 2387 - 2412.

TARIGHATI, A., GROSS, J., AND JALDEN, J. Decentralized Hypothesis Testing in Energy Harvesting Wireless Sensor Networks. IEEE Transactions on Signal Processing 65 , 18 (Sept. 2017), 4862-4873.

TE SUN HAN, AND AMARI, S. Statistical inference under multiterminal data compression. IEEE Transactions on Information Theory 44 , 6 (Oct. 1998), 2300-2324. Number: 6.

TEAM, A. D. P. Learning with privacy at scale.

TENNEY, R. R., AND SANDELL, N. R. Detection with Distributed Sensors. IEEE Transactions on Aerospace and Electronic Systems AES-17 , 4 (July 1981), 501-510. Number: 4.

THORISSON, H. Coupling, Stationarity, and Regeneration . Probability and Its Applications. Springer New York, 2000.

TSITSIKLIS, J. N. Decentralized detection by a large number of sensors. Mathematics of Control, Signals, and Systems 1 , 2 (June 1988), 167-182. Number: 2.

TSYBAKOV, A. B. Introduction to nonparametric estimation . Springer series in statistics. Springer, New York ; London, 2009. OCLC: ocn300399286.

VALIANT, G., AND VALIANT, P. An automatic inequality prover and instance optimal identity testing. SIAM Journal on Computing 46 , 1 (2017), 429-455.

VAN DER HOEVEN, D., HADIJI, H., AND VAN ERVEN, T. Distributed online learning for joint regret with communication constraints. In Proceedings of The 33rd International Conference on Algorithmic Learning Theory (29 Mar-01 Apr 2022), S. Dasgupta and N. Haghtalab, Eds., vol. 167 of Proceedings of Machine Learning Research , PMLR, pp. 1003-1042.

VON MISES, R. Statistik und wahrheit. Julius Springer 20 (1928).

XU, A., AND RAGINSKY, M. Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation. arXiv:1607.00550 [cs, math, stat] (July 2016). arXiv: 1607.00550.

YE, M., AND BARG, A. Optimal schemes for discrete distribution estimation under locally differential privacy. IEEE Transactions on Information Theory 64 , 8 (2018), 5662-5676.

ZAMAN, A., AND SZABÓ, B. Distributed nonparametric estimation under communication constraints. arXiv preprint arXiv:2204.10373 (2022).

ZHAI, C., AND LAFFERTY, J. A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems (TOIS) 22 , 2 (2004), 179-214.

ZHANG, Y., DUCHI, J., JORDAN, M. I., AND WAINWRIGHT, M. J. Information-theoretic lower bounds for distributed statistical estimation with communication constraints. Advances in Neural Information Processing Systems 26 (2013).

ZHU, Y., AND LAFFERTY, J. Distributed nonparametric regression under communication constraints. In Proceedings of the 35th International Conference on Machine Learning (10-15 Jul 2018), J. Dy and A. Krause, Eds., vol. 80 of Proceedings of Machine Learning Research , PMLR, pp. 6009-6017.

A Le Cam theory in distributed setting

We introduce some formal notions of Le Cam theory first in Section A.1. Then, in Section A.2, we study the equivalence of models in the distributed setting. The theoretical developments presented in this section apply to general models denoted by P and Q ; although the main text specifically focuses on the Gaussian location model for P and the multinomial model for Q , the machinery developed here is applicable to general statistical models.

A.1 Preliminary notions of Le Cam theory

A statistical experiment is a set of probability distributions P = { P f : f ∈ F} (a model) on a measurable space ( X , X ) (the sample space). For the purpose of simplification, we shall consider only statistical experiments with Polish sample spaces and corresponding Borel sigma-algebras. Furthermore, we shall only consider dominated models, meaning that there exists a sigma-finite measure µ such that P f ≪ µ for all f ∈ F . In a slight abuse of terminology, we shall sometimes refer to P as the experiment, suppressing the presence of the sample space and indexing set.

Given another statistical experiment with model Q = { Q f : f ∈ F} indexed by the same set F and sample space ( ˜ X , ˜ X ) , we define the deficiency of P with respect to Q as

where the infimum is taken over all Markov kernels C : ˜ X ×X → [0 , 1] and the probability measure P f C : ˜ X → [0 , 1] is understood as

This is equivalent to the more general notion of deficiency of [27] for dominated models on Polish spaces (see Proposition 9.2 in [68]).

The deficiency d ( P ; Q ) quantifies the degree to which Q can be approximated by an experiment P . If d ( P ; Q ) ≤ ϱ , it implies that for bounded loss functions, each decision procedure within Q has an associated procedure in P that achieves nearly the same risk, up to a multiple of ϱ .

To make this precise, let F be a measurable space and consider a function ℓ : F × D → [0 , 1] on a measurable space ( D , D ) , such that t ↦→ ℓ ( f, t ) is measurable for all f ∈ F , which we shall refer to a loss functions . We shall consider a decision procedure for ( Q , D ) to be a Markov kernel D : D × ˜ X → [0 , 1] . If d ( P ; Q ) ≤ ϱ , there exists C : ˜ X ×X → [0 , 1] such that for all decision procedures D for ( Q , D ) we have that

Here, the Markov kernel Q f D is to be understood in the sense of (18) and CD : D ×X → [0 , 1] as

There is also the following reverse implication; suppose that there exists a loss function ℓ : F ×D → [0 , 1] on a measurable space ( D , D ) , and

where the two infimums are over all decision procedures D and Markov kernels C : ˜ X ×X → [0 , 1] . Then, d ( Q , P ) > ϱ . This follows immediately from e.g. Lemma 12 in the appendix, since x ↦→ ∫ ℓ ( f, φ ) dD ( φ | x ) is measurable. In the more extensive framework considered in e.g. [27], such a reverse implication for risk functions fully characterizes the deficiency between two models, but this framework is not needed in what follows.

Le Cam's deficiency distance between P and Q is then defined as

This semi-metric becomes a metric whenever P and Q are identified whenever d ( P ; Q ) + d ( Q , P ) = 0 . Two sequences of experiments P ν and Q ν are called asymptotically equivalent if their difference ∆( P ν , Q ν ) tends to zero as ν approaches infinity. Conversely, such sequences shall be called asymptotically nonequivalent if ∆( P ν , Q ν ) > c as ν →∞ for a fixed constant c > 0 .

The final notion we shall recall is that of sufficiency. A statistic S : X → ˜ X is sufficient for the model P if for any A ∈ X there exists a measurable map ψ A : ˜ X → R such that

Here, the measure P S f is to be understood as the push-forward measure P S f ( B ) = P f ( S -1 ( B )) . A sufficient statistic allows for transforming observations from one model to another, 'sufficient' model which is equivalent in the sense of Le Cam distance. That is, if S is a sufficient statistic for P , then the model P ′ := { P S f : f ∈ F} satisfies ∆( P , P ′ ) = 0 .

The next lemma is the Neyman-Fisher factorization theorem gives a useful characterization of sufficiency of a statistic for models that admit densities with respect to the same dominating measure.

Lemma 2. Suppose that P f ≪ µ for all P f ∈ P with µ a sigma-finite measure. A statistic S : X → ˜ X is sufficient for P if and only if there exists measurable functions g f : R → R and h : X → R such that

A proof for both the lemma and the last statement of the previous paragraph can be found in Chapter 5 of [62].

A.2 Equivalence of distributed decision problems

We now turn to the distributed setting considered in the paper, where j = 1 , . . . , m servers each receive data X ( j ) drawn from a distribution P f and sample space ( X , X ) . Each of the servers communicates a transcript based on the data to a central server, which based on the aggregated transcripts computes its solution to the decision problem at hand.

The tools developed in this section apply to wider range of distributed architectures than the one considered in the main text of the paper, as introduced in Section 2. The framework introduced here accommodates various forms of interaction between servers, including sequential and blackboard protocols (see e.g. [6, 16]). In contrast, the main text focuses on servers that either do not communicate (local randomness protocols) or utilize a shared randomness source (a special case of sequential or blackboard communication).

A distributed protocol for the experiment P with decision space ( D , D ) consists of a triplet { D, K , ( U , U , P U ) } , a Markov kernel D : D × ⊗ m j =1 Y ( j ) → [0 , 1] and a probability space ( U , U , P U ) , and K is a collection of Markov kernels.

To unpack all this notation: the Markov kernel D takes the role of the decision procedure, where the decision is to made on the basis of the transcripts generated by the Markov kernels K . The transcripts are in turn generated based on the data and a source of shared randomness independent of the data. The probability space ( U , U , P U ) plays the role of the source of randomness that is shared by the servers. The distributed protocol is said to have no access to shared randomness or to be a local randomness protocol if U is the trivial sigma-algebra.

In this section, we shall consider three types of communication architectures:

One shot protocols : K = { K j } j =1 ,...,m where K j : Y ( j ) × ( X × U ) → [0 , 1] . These protocols are what are considered in the main text of the paper.

Sequential protocols : K = { K j } j =1 ,...,m where K j : Y ( j ) × ( X × U × Y (1) × · · · × Y ( j -1) ) → [0 , 1] . That is, the transcript generated by server j is based on the data, the shared randomness and the transcripts of the previous servers.

Blackboard protocols :

Blackboard protocols : K = { K j t } j =1 ,...,m,t =1 ,...,T where K j 1 : Y ( j ) × ( X × U ) → [0 ,

for t = 2 , . . . , T . That is, the transcript generated by server j is based on the data, the shared randomness, and the transcripts of all the servers from the previous round.

For one shot protocols, we have in terms of random variables that X ( j ) ∼ P f , U ∼ P U , Y ( j ) | ( X ( j ) , U ) ∼ K j ( ·| X ( j ) , U ) for j = 1 , . . . , m and φ ∼ D ( ·| Y ) with Y = ( Y (1) , . . . , Y ( m ) ) . This gives rise to a Markov chain

For x = ( x (1) , . . . , x ( m ) ) ∈ X m , u ∈ U and { K j } j =1 ,...,m , let x ↦→ K ( A | x ) be the Markov kernel product distribution ⊗ m j =1 K j ( ·| x ( j ) , u ) . Given a distributed protocol and i.i.d. data from P f we shall use P f to denote the joint distribution the data X ∼ P f m , the shared randomness U ∼ P U and Y = ( Y (1) , . . . , Y ( m ) ) with Y | ( X,U ) ∼ K ( Y | X,U ) . We have that P f m K = ⊗ m j =1 P f K j and the push-forward measure of Y then disintegrates as

where the second equality follows from the independence of U with the data X := ( X (1) , . . . , X ( m ) ) drawn from P f .

For sequential protocols, the push-forward measure of Y instead disintegrates as

A one shot or sequential distributed protocol is said to satisfy a b -bit bandwidth constraint if its kernels { K j } j =1 ,...,m are defined on spaces satisfying |Y ( j ) | ≤ 2 b . For blackboard protocols, various bandwidth constraints can be imposed, such as a b -bit bandwidth constraint for each round t = 1 , . . . , T .

Given Markov Kernels C j : X × ˜ X → [0 , 1] , j = 1 , . . . , m , a distributed one shot protocol { D, { K j } j =1 ,...,m , ( U , U , P U ) } for the model P , yields a distributed protocol for the model Q : { D, { CK j } j =1 ,...,m , ( U , U , P U ) } . If { K j } j =1 ,...,m is b -bit bandwidth constraint, the collection of kernels { C j K j } j =1 ,...,m do so too, as each C j K j is defined on Y ( j ) × ˜ X .

Similarly, for a sequential protocol, the Markov kernels C j and K j yield a distributed sequential protocol for the model Q with kernels { C j K j } j =1 ,...,m . If each K j is b -bit bandwidth constraint, so is each C j K j . For blackboard protocols, the same reasoning applies to each round t = 1 , . . . , T . That is, type of protocol defined by the kernels K is 'closed under composition' with kernels C j between ˜ X with target space X , where bandwidth constraints are preserved.

We shall consider the notion of local ϵ -differential privacy of Definition 3. A Markov kernel K : Y ×X → [0 , 1] is called locally ( ϵ, δ ) -differentially private if

Since the definition of differential privacy depends heavily on what one defines as the sample space, it is difficult to obtain a similar 'transfer of distributed protocols' that respects the ( ϵ, δ ) -differential privacy constraint, hence the choice to consider local constraints only.

A one shot or sequential distributed protocol shall be called locally ϵ -differentially private if (23) holds for each K j ; j = 1 , . . . , m . For blackboard protocols, one can impose a ( ϵ, δ ) -differential privacy constraint for each round t = 1 , . . . , T , or for the entire output over t = 1 , . . . , T rounds. The following lemma shows that local ϵ -differential privacy, just like bandwidth constraints, carry over from one model to the other.

Lemma 3. Let ( X , X ) and ( ˜ X , ˜ X ) be measurable spaces and consider Markov kernels C : ˜ X ×X → [0 , 1] and K : Y ×X → [0 , 1] . If K is b -bit bandwidth constraint, so is the Markov kernel CK : Y × ˜ X → [0 , 1] . If K is locally ϵ -differentially private, so is CK . Furthermore, for any collection of Markov kernels K , the same reasoning applies to the collection { CK } K ∈K , preserving bandwidth constraints and local differential privacy, as well as the protocol's architecture.

Proof. The first statement has been remarked on earlier in the section. For the second statement, consider arbitrary ˜ x, ˜ x ′ ∈ ˜ X and A ∈ Y . Using that C is a Markov kernel and applying (23) to K yields

which shows CK is locally ( ϵ, δ ) -differentially private. The above argument applies pointwise for all other conditional arguments in the Markov kernel, hence the same reasoning applies to shared randomness, sequential and blackboard protocols.

In an abuse of notation, let D denote the entire distributed protocol (triplet)

for the experiment P (indexed by F ) with decision space ( D , D ) . Given D and a loss function ℓ : F × D → [ -1 , 1] , we define the distributed risk of D in P for ℓ as

We are now ready to formulate a straightforward consequence for the distributed risk, following from models being close in Le Cam distance. This finding, formulated in Lemma 4, shall serve as one of the main tools for deriving the main results. It states roughly that, whenever there is a b -bit bandwidth constrained distributed protocol that achieves a certain risk is one model and there is small deficiency with the other model relative to the number of servers, there exists a b -bit distributed protocol that achieves comparable risk for the other model. A similar statement holds under local differential privacy constraints. If there is a locally ( ϵ, δ ) -differentially private distributed procedure in the one model and there is small deficiency with another model, it means that there is comparable risk for the privacy constraint distributed decision problem.

Lemma 4. Let m ∈ N . Consider two experiments P and Q with indexing set F , satisfying m d ( Q ; P ) ≤ ϱ for some ϱ > 0 . Let J P and J Q denote the class of b -bit bandwidth constraint shared randomness protocols for the models P and Q respectively.

Then, for any loss function ℓ : F × D → [0 , 1] ,

where in the infimum, in an abuse of notation, D denotes the entire distributed protocol triplet { D, { K j } j =1 ,...,m , ( U , U , P U ) } .

The same statement is holds for J P and J Q denoting either classes of b -bit bandwidth constraint local randomness, sequential protocols, or any of these distributed protocols satisfying local ( ϵ, δ ) -differential privacy constraints, for the respective models P and Q . If Tm d ( Q ; P ) ≤ ϱ , the same statement holds for blackboard protocols with T rounds.

Remark 1 . This exemplifies also that, even though models P m = { P f m : f ∈ F} and Q m = { Q m f : f ∈ F} are close in Le Cam distance, distributed decision problems formulated in terms the models P and Q , can have greatly different performance in terms of associated risks.

Proof. By e.g. Theorem 2 in [62], m d ( Q ; P ) ≤ ϱ implies that there exists a kernel C : X × ˜ X → [0 , 1] such that

By Lemma 3, the kernels ˜ K = { CK : K ∈ K} satisfy the a b -bit bandwidth constraint or local ( ϵ, δ ) -differential privacy constraint if the collection { K j } j =1 ,...,m does. To illustrate this further, consider a one shot distributed protocol for P , { D, { K j } j =1 ,...,m , ( U , U , P U ) } ∈ J P , the distributed protocol ˜ D = { D, { CK j } j =1 ,...,m , ( U , U , P U ) } is then an element of J Q .

Using the fact that ℓ is bounded by one and Lemma 13 in the appendix, it follows that

By Lemma 14 in the appendix,

which combined with (24) finishes for one shot protocols.

Similar reasoning applies to sequential protocols. We start by noting that

̸

Lemma 11 implies that (for any u ) there exists a coupling P of Y (1) ∼ P f K 1 ( · | u ) and ˜ Y (1) ∼ Q f CK 1 ( · | u ) such that 2 P ( Y (1) = ˜ Y (1) ) ≤ ϱ/m . Write

Using the disintegration relationship of (22), we obtain

Iterating the above argument m -1 times, we the statement for sequential protocols. The statement for blackboard protocols follows by combining the above arguments for each round t = 1 , . . . , T .

In the remainder of this text, we shall constrain ourselves to a particular bounded risk function and distributed decision problem; distributed hypothesis testing. The following corollary formalizes the statement at the start of the paragraph for the testing a simple null versus a compositive alternative hypothesis in the distributed setting. To that extent, consider a test of the hypotheses

and an experiment P with indexing set F satisfying { f 0 }∪ H 1 ⊂ F . Consider for m ∈ N a distributed testing protocol for the model P to be a distributed protocol T ≡ { D, { K j } j =1 ,...,m , ( U , U , P U ) } , where in a slight abuse of notation, we shall also use T to denote the (possibly randomized) test T | Y ∼ D ( ·| Y ) . Recalling the notation P Y f = P f m P U K as given in (21), define the distributed testing risk for the hypotheses in (25) and the model P as

Here, D ( T | Y ) := D ( { 1 }| Y ) , but one can equivalently consider a deterministic measurable map T : m Π j =1 Y ( j ) → [0 , 1] without loss of generality. Let T b SR ( P ) (resp. T b LR ( P ) ) denote the set of shared randomness (resp. local randomness) distributed testing protocols for P satisfying a b -bit bandwidth constraint. Similarly, let T ( ϵ,δ ) SR ( P ) (resp. T ( ϵ,δ ) LR ( P ) ) denote the set of shared randomness (resp. local randomness) distributed testing protocols for P satisfying a local ( ϵ, δ ) -differential privacy constraint. Define the same classes for the model Q in the obvious way. Using Lemma 4, we obtain the following result, which is a more general version of Lemma 1.

Lemma 5. Consider experiments P , Q such that m d ( Q ; P ) ≤ ϱ for ϱ > 0 . It holds that

where T is either T b SR , T b LR , T ( ϵ,δ ) SR or T ( ϵ,δ ) LR .

Proof. Given { T, { K j } j =1 ,...,m , ( U , U , P U ) } ∈ T ( P ) , Lemma 4 applied to the loss function and using that { f 0 } ∪ H 1 ⊂ F gives

for some distributed testing protocol { D, { ˜ K j } j =1 ,...,m , ( U , U , P U ) } in T ( Q ) , which yields the first statement. The second statement follows by symmetry of the argument.

The implications of Lemma 4 have implications beyond the testing framework. Whilst in distributed estimation settings, the loss function under consideration is typically not bounded, rates can still be derived in probability. That is, if the minimax rate for the distance d on F in the model P ν is ρ ν , the bounded loss function

ℓ ν ( f, g ) = ✶ { d ( f, g ) ≤ Cρ ν } for C > 0 can be used describe minimax estimation rates (in probability) between models P and Q . Since the paper is about testing, we shall not pursue this direction any further beyond this remark.

In the next sections, we will explore the consequences of Lemma 5 for minimax distributed testing rates for both bandwidth- and privacy constraints.

B Difficulties in direct analysis of the multinomial model under information constraints

Lower-bounds for both estimation and testing problems are typically established by bounding divergence measures between probability distributions, such as the chi-square divergence, mutual information, or total variation; [3, 6, 7, 16 ? , 49, 9, 99, 13, 84, 85, 23, 22].

The proof techniques used for discrete distribution estimation employed in [3, 6] and [7], tight lower-bounds can often be obtained by 'tensorizing' the divergence-breaking the problem into a sum of local divergences. The inferential cost incurred due to bandwidth or privacy constraints are then captured via data processing arguments. Similar tensorization techniques are employed in other estimation problems, see for example [16, 99].

However, this tensorization approach does not yield tight bounds for testing problems. For example, [84] uses mutual information in a tensorization framework for testing but only recovers optimal rates when each server communicates a single bit. Similarly, [8] attempts an estimation-based approach for goodness-of-fit testing but obtains tight lower-bounds only under 1 -bit constraints, as detailed in Section 4 of their paper.

To achieve tight lower-bounds in testing problems, especially under communication and privacy constraints, different techniques are required. Papers [9, 85, 22] employ methods that significantly diverge from those used in estimation. In [9], a combinatorial expansion of the likelihood is used, effective for small sample sizes in the multinomial model but not generalizable to large numbers of

observations. [85] and [22] address this limitation in the Gaussian setting by utilizing the BrascampLieb inequality from functional analysis, which explicitly leverages the Gaussian properties of the log-likelihood. This approach is not directly applicable to discrete models, due to the lack of quadratic structure in the log-likelihood of the multinomial model.

C Separation rates for the Gaussian model

For completeness, we provide the relevant results for the Gaussian model studied in [85] and [22].

The first two results come in the form of lower-bounds for the minimax detection thresholds under bandwidth- and differential privacy constraints for the distributed signal detection problem presented in the introduction. We recall that in this problem, each local machine j ∈ { 1 , . . . , m } observes

with f ∈ R d and Z ( j ) i ∼ N (0 , I d ) , i.i.d. for i = 1 , . . . , n . The null hypothesis constitutes that f = 0 versus the alternative hypothesis that

The following is Theorem 3.1 from [85].

Theorem 4. For each α ∈ (0 , 1) there exists a constant c α > 0 (depending only on α ) such that if

then in the shared randomness protocol case

Similarly, for we have under the local randomness protocol that

Following the proof Theorem 3.1 from [85], we obtain the following lemma.

Lemma 6. Let T b denote the class of b -bit bandwidth constrained shared- or local randomness distributed testing protocols and let ρ satisfy either (28) or (29) , respectively. For any α ∈ (0 , 1) , there exists c α > 0 such that for all T ∈ T ( b ) it holds that

where π = N (0 , c -1 / 2 α d -1 ρ 2 ¯ Γ) for a symmetric, idempotent matrix ¯ Γ ∈ R d × d with d/ 2 ≤ rank ( ¯ Γ) ≤ d .

Similarly, the following result can be derived from the proof of Theorem 5 in [22], by taking s > 0 in the theorem such that d L s = d .

Theorem 5. For each α ∈ (0 , 1) there exists a constant c α > 0 (depending only on α ), such that for any n, m, d ∈ N and

the condition

implies

Similarly, for any n, m, d ∈ N and ϵ, δ satisfying (30) , the condition

implies that

Following its proof, we obtain the following the following sub-result.

Lemma 7. Let T denote the class of sharedor local randomness distributed testing protocols satisfying an ( ϵ, δ ) -differential privacy constraint for 0 < ϵ ≤ 1 , 0 ≤ δ ≤ ( c α m -1 ∧ c α ϵm -1 / 2 ∧ nϵ 2 ∧ n 2 d -1 ϵ 2 ∧ n 3 / 2 d -1 / 2 ϵ 2 ) and let ρ satisfy either (28) or (29) , respectively. For any α ∈ (0 , 1) , there exists c α > 0 such that for all T ∈ T ( ϵ,δ ) it holds that

where π = N (0 , c -1 / 2 α d -1 ρ 2 ¯ Γ) for a symmetric, idempotent matrix ¯ Γ ∈ R d × d with rank ( ¯ Γ) ≍ d .

D Proofs of Theorems 1, 2 and 3

Proof of Theorems 1 and 2. In what follows, let T denote a class of distributed protocols satisfying either a b ≡ b ν -bit bandwidth constraint or a local ( ϵ, δ ) -differential privacy constraint for ϵ ≡ ϵ ν , δ ≡ δ ν , allowing either for shared randomness or only local randomness.

For any sequences m ≡ m ν , d ≡ d ν and n ≡ n ν with C R md log d/ √ n = o (1) , it follows from Lemma 5 and the bound (14) that the testing risks satisfy

Let ρ ∗ ≡ ρ ∗ ν be the minimax rate of the P -distributed problem, over the class T P , in the sense that ρ ∗ equals (up to constants) the right-hand side of (6), (7), (9) or (10). We split the proof into showing that ρ ∗ is an upper and lower-bound for the Q -distributed problem over the class T P .

The rate ρ ∗ is an upper-bound (up to a poly-logarithmic factor) for the minimax rate in Q : Write, for q ∈ F , √ q = ( √ q i ) i ∈ [ d ] . Since X ( j ) - √ q 0 is a sufficient statistic for X ( j ) , the model (13) is equivalent in the Le Cam sense the one generated by

for q ∈ F , which we shall denote by ˜ P . Consequently, by another application of Lemma 5, it suffices to show

If ∥ q -q 0 ∥ 1 ≥ ρ , Lemma 15 implies that ∥ √ q - √ q 0 ∥ 2 ≥ ρ/ 2 . Consequently, if ρ ≡ ρ ν ≫ M ν ρ ∗ where ρ ∗ is of equal order of the minimax rate for the respective class of distributed protocols T P and M ν is an appropriately large factor (of poly-logarithmic order in case of differential privacy constraints), a distributed protocol T ∈ T P exists for the Gaussian model that achieves the separation rate for whenever H 0 : √ q - √ q 0 = 0 versus H ρ : ∥ √ q - √ q 0 ∥ 2 ≥ ρ/ 2 . By the established equivalence of the minimax risks (33), this implies that a protocol T ∈ T Q exists for the multinomial model as well. Thus, ρ ν is an upper-bound for the minimax separation rate for the class of distributed protocols T Q of the multinomial model.

The rate ρ ∗ is a lower-bound for the minimax rate in Q : Suppose that ρ ≡ ρ ν is of smaller order than the minimax rate ρ ∗ of the class T P , in the sense that ρ ∗ /ρ →∞ as ν →∞ . We aim to use the Bayes risk lower-bound of Lemmas 6 and 7, which apply to a Gaussian prior. To accommodate

a Gaussian prior with sufficient mass on the alternative hypothesis, we first need to address the 'constraint on the signal' imposed by ∑ d i =1 q i = 1 for q ∈ F .

To that extent, consider without loss of generality d to be divisible by two. Let I R := [ -( R -1) / ( R + 1) , ( R -1) / ( R +1)] . For all ( f i ) i ∈ [ d/ 2] ∈ I d/ 2 R / √ d , there exists a q f := ( q f i ) i ∈ [ d ] ∈ F such that q f i = 1 /d + f i / √ d for i = 1 , . . . , d/ 2 and q f i = 1 /d -f i / √ d for i = d/ 2 + 1 , . . . , d . To see that q f ∈ F , note that ∑ d i =1 q f i = 1 , q f ≥ 0 and

Define F ′ as the set and

We have F ′ ⊂ F , which in turn implies that H ′ ρ ⊂ H ρ . Combined with the fact that the testing risk decreases by considering smaller alternative hypotheses, this results in

Define g f = (1 / 2)( f, -f ) ∈ R d . By Pinsker's inequality,

where P n √ q - √ q 0 denotes the distribution of (34) and the square root is to be understood as applied coordinate wise.

Let π = N (0 , d -1 ( ρ ∗ ) 2 ¯ Γ) for a symmetric, idempotent matrix ¯ Γ ∈ R d/ 2 × d/ 2 with d/ 4 ≤ rank ( ¯ Γ) ≤ d/ 2 .

We have that

By Lemma 8, the model { P g f : f ∈ I d/ 2 R / √ d } is equivalent to the model generated by the observations for i = 1 , . . . , d/ 2 . Since F ′ is bijective with ( I R / d ) d/ 2 , Lemma 5 implies that

where ˜ P is the model generated by the observations in display (36) for i = 1 , . . . , d/ 2 and P ′ f denotes the distribution of the distributed protocol with data generated from f ∈ ˜ P .

It follows from Lemma 6 in the case of bandwidth constraints or Lemma 7 in the case of privacy constraints (using that ρ ≪ ρ ∗ in both cases) that the latter distributed testing risk is lower-bounded by

Addressing the third term in the display above; the theorem(s) assume that md log d/ √ n ν →∞ → 0 , b ≥ 1 and ϵ ≫ n -1 / 4 , we have that ρ ∗ ≪ 1 / √ log( d ) , which gives ∥ √ df i ∥ ∞ → 0 with π -probability tending to one (see e.g. Lemma 16), which in turn implies that

Next, we show that ∥ ( q f i ) i ∈ [ d ] -q 0 ∥ 1 ≥ ρ with π -probability tending to one. Since ∑ d i =1 | q f i -q 0 | = 2 ∑ d/ 2 i =1 | f i / √ d | , we have that for some constants c, c ′ > 0 ,

where in the expression on the right-hand side, Z ∼ N (0 , I d/ 2 ) . Since ρ ≪ ρ ∗ and ¯ Γ is idempotent with rank of the order d , we can conclude that the expression vanishes. This takes care of the third term in (38).

For the last term in (38), the Taylor approximation √ 1 + y -1 = y/ 2 -y 2 / 8 + y 3 16(1+ η 5 / 2 y ) for some η ∈ [0 , y ] , combined with the fact that ∥ √ df ∥ ∞ = o π (1) yields that

Since the theorem(s) assume that md log d/ √ n ν →∞ → 0 , b ≥ 1 and ϵ ≫ n -1 / 4 , the right-hand side of the above display vanishes.

Proof of Theorem 3. Let T Q , T P denote the class of distributed b -bit bandwidth constrained testing protocols with b ∈ N , m ∈ N , d ∈ 2 N and n ∈ N and no access to shared randomness for the models Q and P , respectively. We note here that under the conditions of the theorem, we can assume d and n are both larger than some constant; and in particular we can assume d ∈ 2 N without loss of generality. Assume d and n satisfy (15), for a constant C to be set later. The proof follows by the fact that the distributed testing problems have different minimax testing rates, for certain values of b and m .

Consider the hypothesis test given in (4), with H 0 : q 0 = (1 /d, . . . , 1 /d ) ∈ S d and H ρ as in the display.

Set b = ⌈ n log 2 ( d ) ⌉ . When b ≥ n log 2 ( d ) , the observations ˜ X ( j ) in the multinomial model as given in (3) are valid b -bit transcripts, since |{ 1 , . . . , d } n | ≤ n log 2 ( d ) . These transcripts are therefore sufficient for the nondistributed / unconstrained model Q m , i.e. corresponding to observations

Consequently, the distributed, b -bit bandwidth constraint testing risk for Q is equal to the testing risk Q m ;

This means that, for all α ∈ (0 , 1) , there exists C α > 0 and a distributed protocol T satisfying a b -bandwidth constraint for distributed experiment Q such that

where H ρ as defined in (4), as the minimax rate for the unconstrained problem with mn observations is ρ 2 Q m := √ d/ ( mn ) (see e.g. Theorem 3 in [70]).

On the other hand, whenever mb = m ⌈ n log 2 ( d ) ⌉ ≤ d , the minimax rate for the distributed testing risk of P for the (comparable) hypotheses

is bounded from below by ρ 2 P ≍ √ d/ ( √ mn ) , as a consequence of Theorem 4. Specifically, following the proof of Theorem 1 above, we have that

where

for I R := [ √ 2(1 -√ R ) / √ 1 + R, √ 2( √ R -1) / √ 1 + R ] , ˜ P is generated by the observations

for Z ( j ) ∼ N (0 , I d/ 2 ) , indexed by f ∈ ( I R / √ d ) d/ 2 and the class T ˜ P is to be understood as the b -bit bandwidth constraint distributed testing protocols for the model ˜ P and j = 1 , . . . , m machines.

Lemma (6) implies that for all α ∈ (0 , 1) the latter is bounded by

for a symmetric, idempotent matrix ¯ Γ ∈ R d/ 2 × d/ 2 with d/ 4 ≤ rank ( ¯ Γ) ≤ d/ 2 α ∈ (0 , 1) , whenever ρ 2 ≤ c α √ d √ mn for some small enough constant c α > 0 . By the same analysis as conducted in the proof of Theorem 1 above (using that n ≤ log( d ) ), we find that the second term is at most α/ 2 for c α > 0 small enough. Summarizing, we find in particular that for some constant c α > 0 ,

for all ρ 2 ≤ c √ d/ ( √ mn ) and m,n,b,d such that mb ≤ d , where the number 1 / 3 is chosen without particular significance.

Whenever mb = m ⌈ n log 2 ( d ) ⌉ ≤ d ,

for some C α > 0 large enough and c α > 0 small enough. Take the constant C = ⌈ C 2 α /c 2 α ⌉ such that if m = C , it holds that

Now suppose that C d ( Q , P ) ≤ 1 / 6 . Lemma 5 then implies in that

This contradicts (40). We conclude that

whenever d/ ⌈ n log 2 ( d ) ⌉ > C . The result now follows with c = 1 / (6 C ) .

E Auxilliary lemmas

The following lemma is used in the comparison of the multinomial model to the many-normal-means model.

Lemma 8. Let d ∈ 2 N , F ⊂ R d/ 2 , and consider for i = 1 , . . . , d independent random variables X i = h i + σZ i with σ > 0 and Z i ∼ N (0 , 1) satisfying

for some f ∈ F and a = ( a i ) i ∈ [ d ] ∈ R d . Let P denote the model generated by the observations X := ( X 1 , . . . , X d ) ∼ P f , f ∈ F and let Q denote the model generated by

with Z i i.i.d. ∼ N (0 , 1) and f ∈ F .

Then, ∆( P , Q ) = 0 .

Proof. The statistic S = ( a i X i -a i X d/ 2+1 ) i ∈ [ d ] is sufficient for the model P by using NeymanFisher (Lemma 2). We have

In distribution, ˜ X = ( ˜ X i ) i ∈ [ d ] is equal to S , which implies ∆( P , Q ) = 0 per Lemma 2.

The following lemmas are well known but included for completeness.

Lemma 9. Let P f denote the distribution of a N ( f, σI d ) distributed random vector for f ∈ R d and let P n f denote the distribution of n i.i.d. draws (i.e. P n f = ⊗ n i =1 P f ).

It holds that

Proof. By Pinsker's inequality,

√

A straightforward calculation gives that the latter is bounded by n 2 σ ∥ f -g ∥ 2 .

The following lemma relates the total variation distance between P, Q to the L 1 -distance between corresponding densities.

Lemma 10. Let P, Q be probability measures dominated by a sigma-finite measure µ with corresponding probability densities p = dP dµ and q = dQ dµ . It holds that

Proof. See e.g. Section 2.4 in [93].

Lemma 11. For any two probability measures P and Q on a measurable space ( X , X ) with X a Polish space and X its Borel sigma-algebra. There exists a coupling P X, ˜ X such that

̸

Proof. See e.g. Section 8.3 in [91].

The next lemma gives a useful characterization of the total variation distance between two probability measures.

Lemma 12. Let P be a signed, bounded measure defined on measurable space ( X , X ) and suppose that P ≪ ν for a sigma-finite measure ν . It holds that

Proof. Consider the Jordan measure decomposition P = P + -P -, where P + , P -are both positive, bounded measures such that P + ⊥ P -. For any measurable f , { f ≥ 0 } , { f ≤ 0 } ∈ X , so | f | ≤ 1 means that

For the other direction, note that f = sign ( p -q ) is measurable and bounded by 1 , which gives

where the last equality follows from Lemma 10.

Lemma 13. Let P = ⊗ m j =1 P j and Q = ⊗ m j =1 Q j for probability measures P j , Q j defined on a common measurable space ( X , X ) , with probability densities p j , q j for j = 1 , . . . m . It holds that

Proof. The measures P j and Q j admit densities with respect to P j + Q j , which we shall denote by p j and q j , respectively, with

Writing µ = ⊗ m j =1 ( P j + Q j ) and applying Lemma 10 we obtain

By the telescoping product identity

and Fubini's Theorem, the right-hand side of (43) is bounded by

The following lemma can be seen as a data processing inequality for the total variation distance.

Lemma 14. Let ( X , X ) and ( Y , Y ) be two measurable spaces and let K : Y ×X → [0 , 1] be a Markov kernel. For any probability measures P, Q defined on X it holds that

Proof. This follows immediately from the representation in Lemma 12 combined with the fact that, for | f | ≤ 1 , x ↦→ ∫ f ( y ) dK ( y | x ) is a measurable function bounded by 1 , since K is Markov kernel. Hence,

The next lemma bounds the L 1 -distance ∥ p -q ∥ 1 between densities with a multiple of the Hellinger distance 2 -1 / 2 ∥ √ p - √ q ∥ 2 .

Lemma 15. For two probability densities p, q with respect to µ , it holds that

Proof. The result follow from the Cauchy-Schwarz inequality and the fact that ∫ pdµ = ∫ qdµ = 1 . See e.g. [93] for details.

Lemma 16. Let K ∈ N and M ∈ R K × K be symmetric and positive definite. Consider the random vector G = ( G 1 , . . . , G K ) ∼ N (0 , M ) . It holds that E max 1 ≤ i ≤ K | G i | ≤ 3 ∥ M ∥ √ log( K ) ∨ log(2) and

for all x > 0 .

Proof. It holds that

Since M is symmetric, positive definite, it has SVD decomposition M = V Diag ( λ 1 , . . . , λ K ) V ⊤ . Since V is orthonormal,

Writing V = [ v 1 . . . v K ] where v k are orthogonal unit vectors, the latter display equals

Hence, it suffices to show that

Consequently,

The case where K = 1 follows by standard Gaussian concentration properties. Assume K ≥ 2 . For 0 ≤ t ≤ 1 / 4 ,

Taking t = 1 / 4 and applying Markov's inequality yields the second statement of the lemma. Furthermore, in view of Jensen's inequality

which in turn yields E max i | Z i | ≤ 3 √ log( K ) .

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: Yes.

Justification: Proofs of all theory are provided.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: Yes.

Justification: See discussion

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: Yes.

Justification:

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: NA

Justification: No code / experiments are provided.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: NA

Justification: No code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: Yes

Justification:

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: NA

Justification: The paper focuses on theoretical developments in statistical models without negative societal applications.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: NA

Justification: All the results are of theoretical nature; no data or models are released.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: NA

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on the minmax rate for goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints in a distributed setting, leveraging Le Cam’s theorem. The main distinction from previous literature lies in the consideration of the distributed setting.\n\nSTRENGTHS:\nThe paper is well-written, well-organized, and mathematically rigorous, with a clear exposition of the concepts using probability theory. The key contribution is the extension of the optimal rate for goodness-of-fit testing under differential privacy to the distributed case. However, I have concerns regarding this extension, which I detail in Weaknesses.\n\nWEAKNESSES:\nI might be wrong, but I didn't see the difference between the distributed formulation in this paper and the central setting in other works (where data is available at one central location). The only exception is that the distributed formulation posits that each server adopts a local protocol (local privacy-mapping with bandwidth constraints). However, given that the raw data \n$X^{(j)},j=1,\\ldots,m$ are i.i.d., wouldn’t the optimal rate be achieved when all local protocols are equivalent? If one server has a better protocol than others, the risk defined after line 190 is not minimized. If this is true, the question is what distinguishes the proposed distributed protocol from the privacy-preserving case in a central setting? For the latter, the min-max rate has already been derived, as seen in:\n\n\"Local Privacy and Statistical Minimax Rates\"\n\n\"Robust Estimation of Discrete Distributions under Local Differential Privacy\"\n\n\"The Cost of Privacy: Optimal Rates of Convergence Performance Estimation with Differential Privacy\"\n\nThe paper needs to clearly articulate the differences and potential advantages of the distributed approach over these established results in the central setting. Without this clarification, the novelty and implications of the results may be unclear.\n\nMy score will change based on the authors' response to this question.\n\nQUESTIONS:\nSee Weaknesses\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores distributed goodness-of-fit testing for discrete distributions under bandwidth and differential privacy constraints. The authors extend results from multivariate Gaussian models using Le Cam’s theory of statistical equivalence. They derive matching minimax upper and lower bounds for the goodness-of-fit testing problem when the number of samples held locally is large.\n\nSTRENGTHS:\nThe framework presented for extending goodness-of-fit testing from Gaussian models to discrete distributions is novel and addresses practical issues in federated learning scenarios. The derivation of matching minimax upper and lower bounds is rigorous and thorough, leveraging statistical equivalence effectively. The paper addresses key challenges in distributed settings, specifically under bandwidth and privacy constraints, which are crucial for modern applications in federated learning.\n\nWEAKNESSES:\nThe results rely on the assumption $md\\log d/\\sqrt{n}=o(1)$, which can be attained when the number of data is large. When $n$ is large, the setting naturally gets close to the Gaussian case, from which some existing tools can be leveraged. In this sense, the analyses presented in this paper are not too surprising. Moreover, the absence of empirical validation or simulations to demonstrate the practical performance of the theoretical results limits the impact of the findings in practice.\n\nQUESTIONS:\n- How practical are the assumption $md\\log d/\\sqrt{n}=o(1)$ in real-world federated learning scenarios? Can you provide examples or case studies where these conditions hold?\n\n- The paper assumes large sample regimes for the derivation of the minimax rates. How would the results change if the sample size was not large? Are there any extensions or modifications of the theory to handle smaller sample sizes?\n\n- How does the proposed method compare with existing methods for distributed goodness-of-fit testing in terms of computational efficiency and communication overhead?\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper investigates the problem of Goodness-of-fit testing for multinomial distributions in federated learning in the case where the number of samples n per federated agent is large, and under a bandwidth or privacy constraint. Under certain scaling regimes, the authors characterize the number of samples needed for risk (defined as sum of Type I and Type II error) to vanish asymptotically. The authors provide an excellent description of Le Cam theory, and discuss it’s relation to their work.

STRENGTHS:
This paper is very well written. It is both thorough and rigorous, while still being approachable and well-written. Section 4 and 5 specifically are very nicely done and explain a complex idea simply. The results are interesting, and novel to my knowledge.

WEAKNESSES:
The paragraphs after Theorem 1 and 2 respectively could be expanded somewhat. It would be interesting to see some discussion about the theorems in context.

Small Comments

166: the the sample

303: citation missing

QUESTIONS:
The title says "Optimal Private *and* Communication Constraint Distributed Goodness-of-Fit Testing". However, it seems that you consider "Optimal Private *or* Communication Constraint", as Theorem 1 and 2 consider these constraints separately. Can you comment on this? What would happen if you consider these jointly?

LIMITATIONS:
I think this paper is thorough, and details the exact theoretical setting where the proven results apply.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses distributed goodness-of-fit testing problems under user-level communication and local differential privacy (DP) constraints. In this scenario, each of the m users receives n samples, and a central server aims to test whether the underlying discrete distribution is uniform. This classical problem has been extensively studied in similar settings, such as when n = 1 or when the task is to estimate the underlying distribution.\n\nThe main contribution of this paper is a tight characterization of the separation rates in the large-sample regime (where n is sufficiently large). The primary technical tool employed in the proof is Le Cam's statistical equivalence. By leveraging the equivalence between the Gaussian location model (GLM) and the multinomial model in large local sample regimes, the problem is reduced to the GLM, which has been addressed in previous works.\n\nWhile Le Cam's statistical equivalence is indeed a powerful and elegant method for establishing minimax rates, I find the novelty and contribution of this work to be somewhat limited. The main technical tools are well-established, and the primary theorem applies only to a restricted parameter regime. Additionally, the discussion of related prior works could be more comprehensive. Lastly, the presentation of the paper could be significantly improved.\n\n============== Post rebuttal =========\nI have updated my score accordingly, given that the authors claim that the techniques used in this work can be extended to prove lower bounds under interactive models.\n\nSTRENGTHS:\nThe main technical tool, the notion of statistical equivalence, seems to be a suitable and powerful method for addressing this class of problems, as it allows for the reduction of one problem to another.\n\nWEAKNESSES:\n1. **Limited Contributions**: The main technical contribution of this paper appears to be limited, as the statistical distance between multinomial models and Gaussian location models is derived from prior works. Additionally, distributed testing for GLM under local DP/communication constraints is also well-established. The main theorems only establish the separation rates in a limited regime (i.e., large \\(n\\)). The proposed reduction, and consequently the statistical equivalence, apply only to the non-interactive setting, and it is well-known that establishing interactive lower bounds is significantly more challenging.\n\n2. **Insufficient Discussion of Prior Works**: There are several relevant works that need further discussion. For instance, the \"communication super-efficiency\" effect, which has appeared in the \"estimation\" version of the problem in [1], is not discussed in the current draft. Although the paper is included in the references, I could not find the corresponding citation in the main text.\n\n3. **Presentation**: The organization and presentation of the paper can be significantly improved. For example, the symbols \\(\\mathcal{Q}\\) and \\(\\mathcal{P}\\) are sometimes used to refer to multinomial and Gaussian models (e.g., in the proof of Theorem 1) without being explicitly stated, and at other times they refer to two general statistical models (e.g., in Section A.2), which is confusing. It would be helpful to explicitly specify these notations. There are also many minor typos and unclear notations that reduce readability. Some examples include:\n   - Line 286: \"identified\" should be \"identical\"?\n   - Line 303: unclear reference []\n   - Line 642: missing reference\n   - Line 673: missing reference\n\n\n[1] Archaya et. al., \"Distributed estimation with multiple samples per user: Sharp rates and phase transition\".\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}