Title: Entropy testing and its application to testing Bayesian networks
Venue: NeurIPS 2024
Accepted: yes
Keywords: bayesian networks

distribution testing

shannon entropy

graphical models

hypothesis testing
Abstract: This paper studies the problem of \emph{entropy identity testing}: given sample access to a distribution $p$ and a fully described distribution $q$ (both are discrete distributions over the support of size $k$), and the promise that either $p = q$ or $ | H(p) - H(q) | \geqslant \varepsilon$, where $H(\cdot)$ denotes the Shannon entropy, a tester needs to distinguish between the two cases with high probability. We establish a near-optimal sample complexity bound of $\tilde{\Theta}(\sqrt{k}/\varepsilon + {1}/{\varepsilon^2}$) for this problem, and show how to apply it to the problem of identity testing for in-degree-$d$ $n$-dimensional Bayesian networks, obtaining an upper bound of $\tilde{O}( {2^{d / 2} n^{3/2}}/{\varepsilon^2} + {n^2}/{\varepsilon^4} )$. This improves on the sample complexity bound of $\tilde{O}(2^{d/2}n^2/\varepsilon^4)$ from Canonne, Diakonikolas, Kane, and Stewart (2020), which required an additional assumption on the structure of the (unknown) Bayesian network.

Entropy testing and its application to testing Bayesian networks

Abstract

This paper studies the problem of entropy identity testing : given sample access to a distribution p and a fully described distribution q (both discrete distributions over a domain of size k ), and the promise that either p = q or | H ( p ) -H ( q ) | ⩾ ε , where H ( · ) denotes the Shannon entropy, a tester needs to distinguish between the two cases with high probability. We establish a near-optimal sample complexity bound of ˜ Θ( √ k/ε +1 /ε 2 ) for this problem, and show how to apply it to the problem of identity testing for in-degreed n -dimensional Bayesian networks, obtaining an upper bound of ˜ O (2 d/ 2 n 3 / 2 /ε 2 + n 2 /ε 4 ) . This improves on the sample complexity bound of ˜ O (2 d/ 2 n 2 /ε 4 ) from [CDKS20], which required an additional assumption on the structure of the (unknown) Bayesian network.

1 Introduction

Entropy is a fundamental information theory notion, which quantifies the amount of 'uncertainty' a given random variable carries. Since its introduction by Shannon, this notion has found myriads of applications, and is central - among others - to compression and coding, probability, electrical engineering, and learning theory.

As a result, the task of estimating the Shannon entropy of a discrete random variable (or, equivalently, its probability distribution) from samples has naturally emerged, starting (in Computer Science) with the work of [BDKR02] which considered multiplicative approximations. Additive approximation of the entropy (within ± ε ) was then considered in a series of papers [VV11a, VV11b, VV13, HJW15a, ADOS17], culminating with the work of [WY16], which establishes the optimal sample complexity, Θ ( k + log 2 k 2 ) , where k ≫ 1 is the domain size.

ε log k ε

While the resulting sample complexity is sublinear in the domain size k , it is only so by a mere logarithmic factor. In some settings, paying this near-linear dependence in the amount of data necessary is impractical, typically in the large-domain regime (e.g., for high-dimensional data, where k is exponential in the dimension); moreover, it may even be unnecessary . Specifically, one may not be concerned so much about the (approximate) value of the entropy of a distribution, but rather about whether it is above a threshold, or differs from that of a given purported model.

It is this latter task we introduce and consider in our work, which can be seen as a variant of the standard identity testing question from distribution testing: given a reference known hypothesis distribution q over a domain of size k , and i.i.d. samples from an unknown distribution p , what is the sample complexity of testing whether p is equal to q , or their entropies differ significantly? And, crucially, is this testing task more sample-efficient than that of estimating H ( p ) ?

Joy Qiping Yang

Entropy Identity testing: Given a reference distribution q , parameter ε > 0 , and samples from an unknown p , what is the cost of deciding (with high probability) whether p = q vs. | H ( p ) -H ( q ) | > ε , with correct probability at least 2 / 3 ?

Note that in the case where q is the uniform distribution over the domain, this task is equivalent to distinguishing between H ( p ) = log k and H ( p ) < log k -ε .

Our main contribution is to show that the testing question can indeed be performed much more efficiently than the estimation one, at least for most parameter regimes. Specifically, we establish the following theorem:

Theorem 1.1. The sample complexity of entropy identity testing is O ( √ k log( k/ε ) /ε +log 2 ( k ) /ε 2 ) . Moreover, this is nearly tight: Ω( √ k/ε +log 2 k/ε 2 ) samples are necessary in the worst case.

Interestingly, this differs both from the estimation task (which, as discussed before, has a near-linear dependence on the domain size k ) but also from identity testing in total variation distance , which has sample complexity Θ( √ k/ε 2 ) (see Section 1.1).

Application: Identity testing for Bayesian networks. As an application of Theorem 1.1, we derive an efficient algorithm for identity testing (in total variation distance) for maximum in-degree d Bayesian networks (shorten as degreed Bayes net in the remaining of the paper): 1

Theorem 1.2 (Informal; see Theorem 3.1) . There is an algorithm which, given sample access to a degreed Bayes net p and the full description of a reference degreed Bayes net q (both over { 0 , 1 } n ), takes ˜ O ( 2 d/ 2 n 3 / 2 ε 2 + n 2 ε 4 ) samples from p , and distinguishes between p = q and d TV ( p, q ) ≥ ε .

Prior to this, the best known sample complexity upper bound for this task [CDKS20] was quadratically worse in both n and ε , and further required an assumption on the underlying graph structure of both p and q . We emphasize that (1) our result improves on the sample complexity of the learning baseline for d ≫ log( n/ε ) , and on its computational efficiency; and (2) compared to the previous testing results, removes strong structural assumptions which considerably limited their applicability. We elaborate on this in the next section.

1.1 Related work

As previously discussed, entropy estimation has received a considerable amount of interest from computer scientists, information theorists and statisticians [BDKR02, Pan04, HJW15a, WY16]. Entropy is also a key example of symmetric property (invariant to relabeling of the domain) [VV11a, VV11b, VV13, ADOS17], and has been considered in other settings as well, e.g., the quantum case [GHS21, AISW20] and the memory-limited setting [ABIS19, AMNW22]. Estimation of some generalizations of Shannon entropy, such as the family of R´ enyi entropies, also have been studied [AOST17].

Over the years, sample complexity of identity testing for discrete distribution has been intensively studied and essentially settled [Pan08, BFF + 01, VV17]. In high dimensions, however, the square root dependence of the sample complexity on the domain size means that most identity testing tasks of interest require sample complexity exponential in the dimension. Moreover, this curse of dimensionality extends to a large range of distribution testing problems [BCY22, Theorem B.1]. As such, many turn to the study of testing distributions under additional natural structural assumptions, such as graphical models: [BGKV21] look at identity testing for product distributions (degree0 Bayes nets) and give the optimal bound of Θ( √ n | Σ | /ε 2 ) , where | Σ | is the alphabet size of each variable (rather than binary alphabet studied in our paper). [DDK19, KDDC23] study testing Ising models, obtaining sample complexity bounds that are poly( n/ε ) ; [DP16], [CDKS20] give tight results to identity testing and closeness testing for a variety of constant in-degree Bayes nets, which also gives polynomial sample complexity bounds.

However, the testing algorithms provided in [CDKS20] and [DP16] are not fully satisfactory, as they require some strong assumptions on Bayes nets. Specifically, [CDKS20, Theorem 21] assumes that

the topological ordering of the two Bayes nets are the same, and shows that under this assumption O (2 d/ 2 n 2 /ε 4 ) samples are sufficient. 2 [CDKS20, Theorem 17] makes the further stringent restriction that the reference Bayes net has to be balanced , i.e., that the conditional probabilities are all bounded away from 0 and 1; moreover, it also requires every parental configuration to be bounded from 0, and that the structure of the unknown Bayes net be a subset of that of the reference one. The result of [DP16, Theorem 4.2] combined with the Hellinger tester from [DKW18, Theorem 1] implies that, under the assumption that p and q share the same factorization structure (i.e., their associated DAGs are the same or one is a subgraph of the other), then this problem is solvable in ˜ O ( 2 d/ 2 n/ε 2 ) samples. While this latter sample complexity is near-optimal (in some regime 3 ), in view of the Ω ( 2 d/ 2 n/ε 2 ) lower bound obtained in [BCY22, Theorem 4.1], the factorization structure requirement considerably limits the applicability of the algorithm.

One can also compare our result to the learning results on Bayesian networks, as any learning algorithms enables testing as well (the 'testing-by-learning' baseline). It is known [CDKS20]) that learning degreed Bayes nets can be done with ˜ O (2 d n/ε 2 ) samples, without any structural assumptions. Our testing result improves on this sample complexity as long as n 2 /ε 4 ≪ 2 d n/ε 2 and 2 d/ 2 n 3 / 2 ≪ 2 d n , i.e., for d ≫ log( n/ε ) ; moreover, it is worth noting that the known learning algorithms are computationally inefficient (running in time n O ( dn ) via an enumeration of all possible underlying graph structures [CDKS20, BGMV20]), and this is believed to be inherent [CHM04]. In contrast, our algorithm runs in time poly( n d , 1 /ε ) .

1.2 Techniques overview

Testing in entropy. A first idea is to use the conversion between total variation (TV) distance and entropy difference to reduce this problem to identity testing in TV : When d TV ( p, q ) ⩽ 1 / 2 , then | H ( p ) -H ( q ) | ⩽ d TV ( p, q ) log k d TV ( p,q ) [CK11, Lemma 2.7]. 4 This gives an upper bound of O ( √ k log 2 ( k/ε ) ε 2 ) , which is already better than the sample complexity of estimation: O ( k ε log k + log 2 k ε 2 ) for the parameter k . However, it is not clear whether the quadratic dependence on ε is necessary: indeed, the 'hard instances' for TV testing (the Paninski construction [Pan08]), small perturbations around the uniform distribution which have TV distance ε from uniform, actually only have entropy log k -Θ( ε 2 ) . The Ω( √ k/ε 2 ) uniformity testing lower bound from these hard instances thus only implies an Ω( √ k/ε ) entropy identity testing lower bound!

A next natural idea is to strengthen the lower bound. However, it then becomes clear that the Paninski [Pan08] construction cannot be improved: as just mentioned, when its TV distance to the uniform distribution is around Θ( √ ε ) its entropy difference to it is only Θ( ε ) (giving an Ω( √ k/ε ) lower bound). Moreover, this is not a coincidence: when the reference distribution q is uniform, we are able to get a matching upper bound using [DKW18, Algorithm 1], upon noticing that

which implies d KL ( p ∥ u k ) = log k -H ( p ) ⩾ ε , where u k is the uniform distribution on [ k ] and d KL denotes the Kullback-Leibler divergence. Interestingly, a completely different hard instance, against a very much non-uniform reference distribution, does yield the second term of our lower bound, Ω(log 2 k/ε 2 ) .

Inspired by these two different lower bounds, we can generalize (1) by defining A as the set of 'not too small probability elements under q ', and then observing (looking ahead, using the inequality (7)) that

where H ( p A ) is the 'entropy' of the sub-distribution restricted to the set A . In particular, this hints that one could solve the general problem by testing if either of the two terms on the right-hand-side

is large. The name of the game now is to (i) choose the threshold for A (i.e., what does it mean for an element to have 'not too small probability under q '), and (ii) have algorithms to test whether these two quantities are noticeably large.

Let us focus on how to test the first term of (2). If min i q i ⩾ Ω ( ε k ) , we can adapt and use an algorithm of [DKW18] to efficiently test d KL ( p ∥ q ) ⩾ ε vs. p = q . In addition, if log(1 /q i ) is bounded, then in fact, estimating the second term to O ( ε ) is possible as well. Thus it is natural to wonder if we can afford to neglect the region where q i ⩽ ε k . Indeed, the impact on entropy is at most O ( τ log( k/τ )) if we are to remove regions with at most O ( τ ) as mass. Thus, by adjusting the appropriate threshold, we can still detect difference in entropy even if we only test on elements with greater than τ/k masses, where τ = ε log( k/ε ) .

The problem then becomes to check if p puts more than 100 τ mass in ¯ A = { i ∈ [ k ] : q i < τ/k } , which costs O (1 /τ ) = O (log( k/ε ) /ε ) samples. If it does, then it cannot be the case that p = q ; we can reject. After this stage, both p ( ¯ A ) , q ( ¯ A ) ⩽ O ( τ ) . To move forward, we need to check the influence on entropy: H ( p ) and H ( q ) . By Jensen's inequality and monotonicity of f ( x ) = x log 1 x when x < 1 e , we have

Therefore, the impact on entropy will be at most O ( τ log k τ ) . Setting τ = ε log( k/ε ) , this becomes O ( ε ) , which gives us the room to check if | H ( p A ) -H ( p A ) | ⩾ 100 ε or p A = q A .

Testing Bayesian networks. Similar to [DP16, Theorem 4.2], 5 the identity testing algorithm is straight-forward: check all every i ∈ [ n ] , if p X i , Π G i = q X i , Π G i or is one of them is far apart, where q is Markov with respect to G ( q factorizes according the DAG G ). The main technical part is to show that the distance is 'subaddititve' when p and q share no common structure, but are close to sharing a common factorization structure (this can be thought of as a relaxtion of [DP16, Theorem 4.2]; refer to Lemma 3.3 for details). As a consequence of 'subadditivity', if p and q are far in distributional distance (differed from [DP16], our work opted to test in KL divergence restricted on subsets with large enough density), then it would imply that one of the local distance between p and q is sufficiently large. This allows us to reduce from global testing to local testing.

Another key aspect is checking whether p and q are close to sharing common structure. More specifically, whether d KL ( p ∥ p G ) is small, where p G is the projection of p unto q 's DAG G . Here, we establish a connection between entropy closeness and structure closeness. In particular, we show that if every local entropy (involving subsets of size d + 1 , where d is the bound on maximum in-degree) between p and q is close, then this means that they must approximately share the same structure (see Lemma 3.4). The intuition behind the connection is that if all tests pass, then we can conclude that p and q are close in local entropy and thereafter, we can utilize entropy of q to learn the graphical structure [KCG + 23] of p (which uses no additional samples).

At a high level, our algorithm first check if p and q roughly share the same structure via a proxy check of local entropy tests. If all local entropy tests pass, then we can show that there exists i ∈ [ n ] such that local KL restricted on subset with large enough mass is greater than Ω ( ε 2 n ) . Asubsequent identity test with χ 2 -test [DKW18] suffice.

Preliminaries and notation. The (Shannon) entropy H of a discrete distribution p supported on [ k ] is given by:

The conditional entropy H ( p X | p Y ) for X supported on X , and Y on Y , defined by the joint distribution p X,Y , can be written as

We adopt the entropy notation for a sub-probability vector H ( q A ) = ∑ i ∈A q i log 1 q i . Throughout this paper, we will use e as base of the log and of the entropy. We will use ← for variable assignment. Weadopt the standard O ( · ) , Ω( · ) and Θ( · ) asymptotic notation and use ˜ · to hide any polylogarithmic factors in the argument. We will use various metrics or divergences on probability distributions: Kullback-Leibler ( d KL ), Hellinger ( d H ), chi-squared ( d χ 2 ), and total variation ( d TV ). We denote p A as restricting p onto the elements in A , and we denote distributional distances restricting on A ∑ √ ∑ √ 2

p

q

i

∈A

p

i

log

i

i

.

d

H

(

p

A

, q

A

) =

1

2

i

∈A

p

i

- √

q

i

.

For a set

. We also have the following inequality [DKW18, Proposition 1]:

as follows:

A

, we write

d

KL

p

(

(

p

A

, q

A

) =

A

) =

∑

i

∈A

p

i

A distribution p supported over the hypercube { 0 , 1 } n is a Bayesian network if its probability mass function satisfies the factorization associated with G , a directed acyclic graph (DAG):

and Π i is the set of parents of X i in G ; and we say that p is Markov with respect to DAG G . In section 3, slightly abusing notation, we use p G to denote a projection of a Bayes net p to a DAG G (which it may or may not be Markov with respect to; see Definition 3.2). We work in the Poissonized setting (see, e.g., [Can22, Appendix C]) - instead of drawing N samples directly from p , we draw Y ∼ Poi( N ) samples from p , where Poi( N ) denotes the random variable distributed as the Poisson distribution with parameter N . The Poissonized and usual sampling settings are equivalent for constant probability of failure, up to a (small) multiplicative factor in the sample complexity.

2 Near-optimal entropy testing

We prove Theorem 1.1, establishing the sample complexity upper and lower bounds separately.

We will prove the following theorem:

Theorem 2.1. There is an algorithm (Algorithm 1) which, given n samples from a discrete distribution p , the full description of a reference distribution q , both over [ k ] , and parameter ε > 0 , distinguishes between p = q and | H ( p ) -H ( q ) | ⩾ ε with probability at least 2 / 3 , as long as

and c 2 ε ⩽ k , for some absolute constants c 1 , c 2 > 0 . Moreover, the algorithm runs in time linear in the number of samples n and the domain size k .

The proof will rely on the two following claims and Lemma 2.4, which is a straightforward extension of [DKW18, Lemma 2]. Their proofs are deferred to Appendix B. Throughout, we let τ := ε 16 log( k/ε ) , and A := { i ∈ [ k ] | q i ⩾ τ k } , as in Algorithm 1.

Claim 2.2. Let A be any set such that p ( ¯ A ) < ε/ 2 . Then, if | H ( p A ) -H ( q A ) | ⩾ ε , we must have (i) d KL ( p A ∥ q A ) ⩾ ε 2 or (ii) | ∑ i ∈A ( p i -q i ) log( 1 q i ) | ⩾ ε 2 .

Claim 2.3. Let ˆ p be the empirical estimator for an unknown discrete distribution p supported on [ k ] , based on Poi( m ) samples, where m = Θ ( log 2 ( k ) ε 2 ) ; assume that d χ 2 ( p A , q A ) ⩽ ε/ 8 and p ( ¯ A ) + q ( ¯ A ) ⩽ 4 τ = 1 4 ε log( k/ε ) , 6 then

√

(

)

Algorithm 1 Entropy identity testing

Require: Sample access to p and full description of q , both over [ k ] ; accuracy parameter ε .

1: Set τ := ε 16 log( k/ε ) , and A := { i ∈ [ k ] | q i ⩾ τ k } .

2: Take m 1 = 48 /τ samples from p and compute the empirical ˆ p ′ .

3: Compute Z 1 = ˆ p ′ ( ¯ A ) .

4: if Z ⩾ 2 τ then return reject

1 ▷ Early rejection.

▷ N i : the empirical count among samples of the i -th element. √ and compute

5: Let m 2 = 65536 ( k · log( k/ε ) ε ) . Draw Poi( m 2 ) samples from p

6: if Z 2 ⩾ 1 16 m 2 ε then return reject

7: Let m 3 = 140800 ( log 2 ( k ) ε 2 )

8: Draw Poi( m 3 ) samples from p , compute the empirical ˆ p ; let Z 3 ← ∣ ∣ ∣ ∑ i (ˆ p i -q i ) log ( 1 q i )∣ ∣ ∣ .

9: if Z 3 ⩾ 1 8 ε then return reject

10: return accept

Lemma 2.4. Let A := { i ∈ [ k ] | q i ⩾ α } . Let m 2 ⩾ 16384 max {√ 1 αε , √ k ε } be the number of samples used to compute Z 2 . Then E [ Z 2 ] = m 2 d χ 2 ( p A , q A ) . Moreover, if d χ 2 ( p A , q A ) ⩽ ε 2 , then Var[ Z 2 ] ⩽ ( 1 32 m 2 ε ) 2 . If d χ 2 ( p A , q A ) ⩾ ε , then Var[ Z 2 ] ⩽ O ( E [ Z 2 ] 2 ) .

Proof of Theorem 2.1. We prove the statement by analyzing Algorithm 1. First, note that excluding the set of ¯ A (elements with small mass), can change the value of H ( q ) by at most ε/ 8 : indeed, by Jensen's inequality ( f ( x ) = log x is concave) and x log 1 x being monotonically increasing in (0 , 1 /e ) ,

i ∈ ¯ A ε, when τ ⩽ 1 /e . Similarly, if p ( ¯ A ) ⩽ 3 τ , we have that H ( p ¯ A ) ⩽ 3 8 ε . Therefore, |

For Line 4, we prove the following: with probability at least 99 / 100 , if Z 1 ⩾ 2 τ , then p ( ¯ A ) ⩾ τ ; and if Z 1 < 2 τ , then p ( ¯ A ) < 3 τ (this is a standard technique; see e.g., [Can22, Fact 2.2].) For the sake of completeness we include the full derivation in the Appendix A.

After Line 4 of Algorithm 1. We conclude from the above that

A still has sufficient entropy gap to test on: | H ( p A ) -H ( q A ) | ⩾ 1 2 ε .

With probability at least 99 / 100 , when p = q , it will not be rejected in Algorithm 4 of Line 4; and once it is pass through this stage, we have p ( ¯ A ) ⩽ 3 τ .

Completeness: when p = q .

We have that d χ 2 ( p A , q A ) = 0 , and via Lemma 2.4, we know that E [ Z 2 ] = 0 and Var[ Z 2 ] ⩽ 1 32 2 m 2 2 ε 2 . By Chebyshev's inequality,

16 4

On the other hand, by Claim 2.3, setting m 3 = 140800 log 2 ( k ) ε 2 , we have that with probability at least 99 / 100 ,

Therefore, with probability at least 1 -1 4 -2 100 = 73 100 > 2 3 , the tester will accept.

Soundness: when | H ( p ) -H ( q ) | ⩾ ε . If p ( ¯ A ) ⩾ 3 τ then ˆ p ( ¯ A ) ⩾ 2 τ with probability 99 / 100 , and the algorithm will output Reject . We proceed assuming p ( ¯ A ) ⩽ 3 τ and recall Item ii. from before, we have | H ( p A ) -H ( q A ) | ⩾ 1 2 ε . By Claim 2.2, we have that either d KL ( p A , q A ) ⩾ 1 4 ε or ∣ ∣ ∑ i ∈A ( p i -q i ) log (1 /q i ) ∣ ∣ ⩾ 1 4 ε . We apply Lemma 2.4, setting α = τ/k and m 2 ⩾ 65536 √ k log( k/ε ) /ε .

If d KL ( p A ∥ q A ) ⩾ 1 4 ε , with (4) and exp(3 / 2) ⩽ k/ε , we have

which by Lemma 2.4, and our setting of m 2 and α , implies Var[ Z 2 ] ⩽ ( 1 4 E [ Z 2 ]) 2 and E [ Z 2 ] = m 2 · d χ 2 ( p A , q A ) ⩾ 1 8 m 2 ε . By Chebyshev,

On the other hand, if it is the case that ∣ ∣ ∑ i ∈A ( p i -q i ) log (1 /q i ) ∣ ∣ ⩾ 1 4 ε , by Claim 2.3, setting m 3 = 140800log 2 ( k ) /ε 2 , with probability at least 99 / 100 ,

We have that Z 3 = ∣ ∣ ∣ ∑ i (ˆ p i -q i ) log 1 q i ∣ ∣ ∣ ⩾ 1 8 ε and thus with probability at least 1 -1 4 -2 100 = 73 100 , the following will happen, the tester will reject: either p ( ¯ A ) ⩾ 3 τ , and it is rejected at Line 4 of Algorithm 4, or it passes and p ( ¯ A ) ⩽ 3 τ and

and will be rejected. This concludes the proof.

Remark 2.5. We note that we can slightly improve the sample complexity of Theorem 1.1 (specifically, improving on the √ k log( k/ε ) term), at the price of a more complicated algorithm, by adding thresholds τ ′ = ε log log( k/ε ) , τ ′′ = ε log log log( k/ε ) , and considering separately the elements in A ′ = { i : q i ∈ ( τ/k, τ ′ /k ] } , A ′′ = { i : q i ∈ ( τ ′ /k, τ ′′ /k ] } ; specifically, by grouping them in groups, and 'merging' each group to get a 'new' element with larger probability. For the sake of clarity, we defer this improvement to Appendix D.

2.2 An Ω( √ k/ε +log 2 k/ε 2 ) lower bound

The Ω( √ k/ε + log 2 k/ε 2 ) lower bound comes from the combination of Lemma 2.6 and Lemma 2.7. We obtain Lemma 2.6 through the classical hard instance used for uniformity testing [Pan08] and a simple conversion between TV distance and entropy difference gives the result. We note that distributions close to uniform distribution actually have smaller entropy difference (uniform distribution is quite special: having the highest entropy of log k ). Indeed, replacing the uniform distribution with a slightly biased distribution, we obtain another hard instance for Lemma 2.7, using the classical Le Cam's two-point method.

Algorithm 2 Identity testing for bounded degree Bayes nets

Require: Sample access to Bayes net p , full description of Bayes net q , accuracy parameter ε , in-degree d and dimension n .

3: for all L ∈ N d +1 ∪ N d do ▷ N ℓ is all subsets of { 0 , 1 } n with size ℓ

4: Call Algorithm 1 with p L , q L and S 1 ; ▷ Entropy test on p L and q L with accuracy ε 2 /n .

5: if Entropy test rejects then return reject

6: S 3 ← O ( n 2 · log( n/ε ) · d log( n ) ε 2 ) samples from p and compute its empirical distribution ˆ p ;

7: for all i ∈ [ n ] do

9: Check d KL ( p X i , Π G i ; A ′ i ∥ q X i , Π G i ; A ′ i ) ⩾ ε 2 n or d KL ( p X i , Π G i ; A ′ i ∥ q X i , Π G i ; A ′ i ) = 0 .

8: if ˆ p X i , Π G i ( ¯ A ′ i ) ⩾ Ω( ε 2 / ( n 2 · log( n/ε ))) then return reject

10: if i -th KL test says far then return reject

11: return accept

▷ Accept if all tests pass.

Lemma 2.6. With fewer than c 3 · √ k/ε samples from p , no tester can distinguish between p = q and | H ( p ) -H ( q ) | ⩾ ε with probability higher than 2 / 3 , where c 3 > 0 is an absolute constant.

Lemma 2.7. With fewer than c 4 · log 2 k/ε 2 samples from p , no tester can distinguish between p = q and | H ( p ) -H ( q ) | ⩾ ε with probability higher than 2 / 3 , where c 4 > 0 is an absolute constant.

3 Application to identity testing for Bayes nets

We now provide an application of our main entropy identity testing theorem, to obtain an improved 'standard' identity testing algorithm for Bayesian networks:

Theorem 3.1. Given sample access to an in-degree d Bayes net p and full description of in-degree d Bayes net q , Algorithm 2 takes

samples to test between p = q or d H ( p, q ) ⩾ Ω( ε ) , where C > 0 is an absolute constant. Moreover, the algorithm runs in time polynomial in n d and 1 /ε .

Before proceeding to the analysis of our algorithm, we require the following definitions.

Definition 3.2. A projection of a Bayes net p on { 0 , 1 } n unto a DAG G is denoted p G , and is defined by its probability mass function (PMF) as follows:

where Π G i is the set of parents of X i in G . Abusing the notation in the context of Bayesian networks, we refer to p X i , Π i or p X i , Π i ( x i , π i ) as the marginal distribution of p on the subset { X i , Π i } .

This gives us the property that marginalization over X i = x i , Π G i = π G i works nicely as we include elements only based on its local property (as long as q X i , Π i is large enough). And q is Markov w.r.t.

Let ( a, B ) ∈ A ′ i , we have that as long as ( x i ( x ) , π i ( x )) = ( a, B ) , then x ∈ A i and vice versa, which means that

We will check if p X i , Π G i ( ¯ A ′ i ) ⩾ Ω( ε 2 / ( n 2 · log( n/ε ))) and reject early if true; this takes O ( n 2 · log( n/ε ) · d log( n ) ε 2 ) samples for all tests to be correct via a union bound. After passing this test, we can conclude that

where we marginalize over everything other than ( X 1 , Π G 1 ) in the third step.

Similarly, we can upper bound q ( ¯ U ) ⩽ q X i , Π G i ( ¯ A ′ i ) ⩽ O ( ε 2 / ( n 2 · log( n/ε ))) . Abusing the notation slightly, we denote p G ; U as the distribution obtained by projecting p onto G (which gives p G ) and then restricting the distribution p G to take elements in U .

We will need the following Lemma 3.3 whose proof is deferred to Appendix E.

Lemma 3.3. Suppose d 2 H ( p, q ) ⩾ Ω( ε 2 ) ; d KL ( p ∥ p G ) ⩽ O ( ε 2 ) ; p ( ¯ U ) ⩽ ε 2 n log( n/ε ) ; ∀ i ∈ [ n ] , p ( ¯ A ′ i ) ⩽ ε 2 n 2 log( n/ε ) , where A ′ i is defined above, and q is Markov with respect to G , then we have

Therefore testing d KL ( p X i , Π G i ; A ′ i ∥ q X i , Π G i ; A ′ i ) ⩾ ε 2 n over all i suffices to detect this case.

In addition, to connect entropy testing to Bayes net testing, we will need the following Lemma 3.4, whose proof will be deferred to Appendix E. And so if we can show that all local entropies between Bayes nets p and q are sufficiently close, then this implies that p must be close to q 's DAG G and q must also be close to p 's DAG G ′ .

Lemma 3.4. Let p and q be two max in-degreed Bayes nets supported on { 0 , 1 } n such that for every subset L ⊆ { X 1 , · · · , X n } of size d +1 , the following holds:

Suppose p is Markov w.r.t. G ′ and q Markov w.r.t. G . Then we have that

Proof of Theorem 3.1. We show the result by analyzing Algorithm 2. By Theorem 1.1, the sample complexity for entropy testing on any subset L of size (dimension) d or d +1 , is

To guarantee the success of every tests employed in the algorithm, we increase the sample complexity of each test by an extra O (log( n d +1 )) = O ( d log n ) factor to boost their success probability to 1 -1 100 n d +1 (via a standard majority vote technique), which will allow us to use a union bound over all tests as there are at most n d +1 subsets with size d +1 . For this step, the sample complexity will be

With this in hand, we will proceed with the analysis under the event that every entropy test performed is correct (which by the above argument happens with high probability). If distribution p manages to pass all the entropy tests, it must satisfy the following:

for every subset L of size d +1 for the latter, and every subset L of size d or d +1 for the former. From here, in principle, we can perform structural learning of p through H ( q L ) , which then gives us an approximated DAG ˆ G of p and we can check d H ( p ˆ G , q ˆ G ) . Unfortunately, structural learning

of Bayes nets is known to be computationally hard in many settings [H¨ of93, CHM04], and so this would lead to a computationally inefficient algorithm.

Instead, we argue that this (learning) step can be bypassed entirely: the intuition of the argument is to view structure learning for Bayes net as an optimization problem; and any assignment x to the two optimization problems (structure learning of p and q ) satisfy f 1 ( x ) = f 2 ( x ) ± O ( ε 2 ) due to their local entropy being close 7 - this means that an optima x 1 for f 1 satisfies min x f 1 ( x ) = f 1 ( x 1 ) ⩾ f 2 ( x 1 ) -ε 2 ⩾ min x f 2 ( x ) -ε 2 and vice versa (optima x 2 for f 2 ).

Applying Lemma 3.4, we have that d KL ( p ∥ p G ) ⩽ O ( ε 2 ) where G is the DAG q is Markov with respect to. With this at hand, we continue onto the KL testing part. The algorithm will check if p X i , Π G i ( ¯ A ′ i ) ⩾ Ω( ε 2 / max( n, log(1 /ε ))) and reject early if it is true (this costs

Recalling (4), if the former case holds, we have

the bound on the first term following from the algorithm's check on Line 10. Using Lemma 2.4, we can perform the corresponding check for i ∈ [ n ] , and after a union bound over n tests, noting that q ( x i , π G i ) ⩾ ε 2 2 d +1 n 2 log( n/ε ) when restricted on A ′ i , the sample complexity is

Following this, we look at the two cases:

If p = q , then with high probability, p will pass all entropy tests, all KL local tests and the tester will accept.

If d H ( p, q ) ⩾ ε , either it fails one of the entropy tests. If it does pass the entropy test, then we must have that d KL ( p ∥ p G ) ⩽ O ( ε 2 ) by (26). Then following Lemma 3.3 and Lemma 2.4, the tester will reject.

In total, the sample complexity is:

This concludes the proof of the theorem.

4 Conclusion and open problems

In this paper, we study a variant of distribution testing problem in terms of entropy difference; we give nearly tight upper and lower sample complexity bounds for the problem. We subsequently apply our entropy testing algorithm to identity testing of Bayes nets, which unlike prior works, makes merely the necessary assumptions (the bound on the in-degree of the Bayes nets).

Future directions. We believe the closeness (two-sample) testing variant of the problem (testing if two unknown distribution p and q are the same or far in terms of entropy difference) could also be interesting; and, notably, has connections to other distribution testing problems: first, it should lead to a natural solution to closeness testing of Bayes nets via ideas in this paper. Second, solving the closeness entropy testing problem give another path to testing independence in terms of mutual information (studied in [BGP + 23] and also covered in [CDKS18]), a notion closely related to entropy.

Acknowledgments and Disclosure of Funding

We would like to thank the reviewers for their suggestions and efforts which help improve this paper.

References

[ABIS19] Jayadev Acharya, Sourbh Bhadane, Piotr Indyk, and Ziteng Sun. Estimating entropy of distributions in constant space. In NeurIPS , pages 5163-5174, 2019.

[ADOS17] Jayadev Acharya, Hirakendu Das, Alon Orlitsky, and Ananda Theertha Suresh. A unified maximum likelihood approach for estimating symmetric properties of discrete distributions. In ICML , volume 70 of Proceedings of Machine Learning Research , pages 11-21. PMLR, 2017.

[AISW20] Jayadev Acharya, Ibrahim Issa, Nirmal V. Shende, and Aaron B. Wagner. Estimating quantum entropy. IEEE J. Sel. Areas Inf. Theory , 1(2):454-468, 2020.

[AMNW22] Maryam Aliakbarpour, Andrew McGregor, Jelani Nelson, and Erik Waingarten. Estimation of entropy in constant space with improved sample complexity. In NeurIPS , 2022.

[AOST17] Jayadev Acharya, Alon Orlitsky, Ananda Theertha Suresh, and Himanshu Tyagi. Estimating renyi entropy of discrete distributions. IEEE Trans. Inf. Theory , 63(1):38-56, 2017.

[BCY22] Arnab Bhattacharyya, Cl´ ement L. Canonne, and Joy Qiping Yang. Independence testing for bounded degree bayesian network. CoRR , abs/2204.08690, 2022.

[BDKR02] T˘ ugkan Batu, Sanjoy Dasgupta, Ravi Kumar, and Ronitt Rubinfeld. The complexity of approximating entropy. In STOC , pages 678-687. ACM, 2002.

[BFF + 01] Tugkan Batu, Lance Fortnow, Eldar Fischer, Ravi Kumar, Ronitt Rubinfeld, and Patrick White. Testing random variables for independence and identity. In FOCS , pages 442-451. IEEE Computer Society, 2001.

[BGKV21] Arnab Bhattacharyya, Sutanu Gayen, Saravanan Kandasamy, and NV Vinodchandran. Testing product distributions: A closer look. In Algorithmic Learning Theory , pages 367-396. PMLR, 2021.

[BGMV20] Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, and N. V. Vinodchandran. Efficient distance approximation for structured high-dimensional distributions via learning. In NeurIPS , 2020.

[BGP + 23] Arnab Bhattacharyya, Sutanu Gayen, Eric Price, Vincent Y. F. Tan, and N. V. Vinodchandran. Near-optimal learning of tree-structured distributions by chow and liu. SIAM J. Comput. , 52(3):761-793, 2023.

[BY02] Ziv Bar-Yossef. The Complexity of Massive Data Set Computations . PhD thesis, UC Berkeley, 2002. Adviser: Christos Papadimitriou. Available at http://webee. technion.ac.il/people/zivby/index_files/Page1489.html .

[Can22] Cl´ ement L. Canonne. Topics and techniques in distribution testing: A biased but representative sample. Found. Trends Commun. Inf. Theory , 19(6):1032-1198, 2022.

[CDKS18] Cl´ ement L. Canonne, Ilias Diakonikolas, Daniel M. Kane, and Alistair Stewart. Testing conditional independence of discrete distributions. In STOC , pages 735-748. ACM, 2018.

[CDKS20] Cl´ ement L. Canonne, Ilias Diakonikolas, Daniel M. Kane, and Alistair Stewart. Testing bayesian networks. IEEE Trans. Inf. Theory , 66(5):3132-3170, 2020. Preprint available at arXiv:1612.03156.

[CHM04] David Maxwell Chickering, David Heckerman, and Christopher Meek. Large-sample learning of bayesian networks is np-hard. J. Mach. Learn. Res. , 5:1287-1330, 2004.

[CK11] Imre Csisz´ ar and J´ anos K¨ orner. Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition . Cambridge University Press, 2011.

[CL68] C. K. Chow and C. N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory , 14(3):462-467, 1968.

[DDK19] Constantinos Daskalakis, Nishanth Dikkala, and Gautam Kamath. Testing ising models. IEEE Trans. Inf. Theory , 65(11):6829-6852, 2019.

[DKW18] Constantinos Daskalakis, Gautam Kamath, and John Wright. Which distribution distances are sublinearly testable? In Proceedings of the Twenty-Ninth Annual ACMSIAM Symposium on Discrete Algorithms , pages 2747-2764. SIAM, 2018.

[DP16] Constantinos Daskalakis and Qinxuan Pan. Square hellinger subadditivity for bayesian networks and its applications to identity testing. arXiv preprint arXiv:1612.03164 , 2016. Full version of [DP17].

[DP17] Constantinos Daskalakis and Qinxuan Pan. Square hellinger subadditivity for bayesian networks and its applications to identity testing. In COLT , volume 65 of Proceedings of Machine Learning Research , pages 697-703. PMLR, 2017.

[GHS21] Tom Gur, Min-Hsiu Hsieh, and Sathyawageeswar Subramanian. Sublinear quantum algorithms for estimating von neumann entropy. CoRR , abs/2111.11139, 2021.

[HJW15a] Yanjun Han, Jiantao Jiao, and Tsachy Weissman. Adaptive estimation of shannon entropy. In ISIT , pages 1372-1376. IEEE, 2015.

[HJW15b] Yanjun Han, Jiantao Jiao, and Tsachy Weissman. Adaptive estimation of shannon entropy. CoRR , abs/1502.00326, 2015.

[H¨ of93] Klaus-U H¨ offgen. Learning and robust learning of product distributions. In Proceedings of the sixth annual conference on Computational learning theory , pages 77-83, 1993.

[KCG + 23] Neville Kenneth Kitson, Anthony C. Constantinou, Zhigao Guo, Yang Liu, and Kiattikun Chobtham. A survey of bayesian network structure learning. Artif. Intell. Rev. , 56(8):8721-8814, 2023.

[KDDC23] Anthimos Vardis Kandiros, Constantinos Daskalakis, Yuval Dagan, and Davin Choo. Learning and testing latent-tree ising models efficiently. In COLT , volume 195 of Proceedings of Machine Learning Research , pages 1666-1729. PMLR, 2023.

[Pan04] Liam Paninski. Estimating entropy on m bins given fewer than m samples. IEEE Trans. Inform. Theory , 50(9):2200-2203, 2004.

[Pan08] Liam Paninski. A coincidence-based test for uniformity given very sparsely sampled discrete data. IEEE Transactions on Information Theory , 54(10):4750-4755, 2008.

[VV11a] Gregory Valiant and Paul Valiant. Estimating the unseen: an n/log(n)-sample estimator for entropy and support size, shown optimal via new clts. In STOC , pages 685-694. ACM, 2011.

[VV11b] Gregory Valiant and Paul Valiant. The power of linear estimators. In FOCS , pages 403-412. IEEE Computer Society, 2011.

[VV13] Paul Valiant and Gregory Valiant. Estimating the unseen: Improved estimators for entropy and other properties. In NIPS , pages 2157-2165, 2013.

[VV17] Gregory Valiant and Paul Valiant. An automatic inequality prover and instance optimal identity testing. SIAM J. Comput. , 46(1):429-455, 2017.

[WY16] Yihong Wu and Pengkun Yang. Minimax rates of entropy estimation on large alphabets via best polynomial approximation. IEEE Trans. Inform. Theory , 62(6):37023720, 2016.

A Derivation for Line 4 in Algorithm 1

We need to show the following: with probability at least 99 / 100 , if Z 1 ⩾ 2 τ , then p ( ¯ A ) ⩾ τ ; and if Z 1 < 2 τ , then p ( ¯ A ) < 3 τ . For the first one, we prove by contrapositive: with high probability 1 -1 200 , p ( ¯ A ) < τ ⇒ Z 1 < 2 τ . Suppose T = Binomial( m 1 , τ ) and setting m 1 = 48 τ ⩾ 3 log 200 τ , and using a Chernoff bound, we have the following

Since any Binomial( m 1 , p ( ¯ A )) will be first-order stochastic dominated by Binomial( m 1 , τ ) if p ( ¯ A ) < τ , we can conclude the following: if p ( ¯ A ) < τ , then Pr[ Z 1 ⩾ 2 τ ] ⩽ Pr[ T ⩾ 2 τ ] ⩽ 1 200 .

For the latter, we prove via its contrapositive: with probability 1 -1 200 , p ( ¯ A ) ⩾ 3 τ ⇒ Z 1 ⩾ 2 τ . As p ( ¯ A ) ⩾ 3 τ , take m 1 = 48 τ ⩾ 9 log 200 τ , by a Chernoff bound, we have

Combining the two with a union bound concludes the proof.

B Deferred proofs from Section 2.1

Claim 2.2. Let A be any set such that p ( ¯ A ) < ε/ 2 . Then, if | H ( p A ) -H ( q A ) | ⩾ ε , we must have (i) d KL ( p A ∥ q A ) ⩾ ε 2 or (ii) | ∑ i ∈A ( p i -q i ) log( 1 q i ) | ⩾ ε 2 .

Proof of Claim 2.2. We can bound | H ( p A ) -H ( q A ) | as

which implies that at least one of the two terms is at least ε/ 2 . If it is the second, we are done; otherwise, we know that either

We will rule out the second case, using that log 1 x ⩾ 1 -x for x > 0 , 8

Thus, we cannot have d KL ( p A ∥ q A ) ⩽ -1 ε and so d KL ( p A ∥ q A ) ⩾ 1 ε .

2 2

Claim 2.3. Let ˆ p be the empirical estimator for an unknown discrete distribution p supported on [ k ] , based on Poi( m ) samples, where m = Θ ( log 2 ( k ) ε 2 ) ; assume that d χ 2 ( p A , q A ) ⩽ ε/ 8 and p ( ¯ A ) + q ( ¯ A ) ⩽ 4 τ = 1 4 ε log( k/ε ) , 9 then

Proof of Claim 2.3. We follow the same analysis as in [WY16]. Letting Y i := ( p i -ˆ p i ) log 1 q i for i ∈ A , we have E [ Y i ] = 0 and

Let Y := ∑ i ∈A ( p i -ˆ p i ) log 1 q i . We will use our assumption that d χ 2 ( p A , q A ) ⩽ ε/ 8 and p ( ¯ A ) + q ( ¯ A ) ⩽ 1 4 ε log( k/ε ) below. Note that, our analysis is in the Poissonized setting:

For (8), we analyze by two cases: if p i q i ⩾ 1 , we have that p i log 2 ( p i q i ) ⩽ p i ( p i q i -1 ) ; otherwise, p i log 2 ( p i q i ) = p i log 2 ( q i p i ) < p i ( q i p i -1 ) . And we use [HJW15b, Lemma3], ∑ i ∈A p i log 2 ( 1 p i ) ⩽ 2 log 2 k +3 in (9). We use the premise and (4) in (10) and we have that

and the last step is obtained by noticing that log( k ) ⩾ 2 3 for k ⩾ 2 . By Chebyshev's inequality, we then have that

and this last inequality yields the statement as long as m ⩾ 22 × 100 × 8 2 log 2 ( k ) ε 2 = 140800 log 2 ( k ) ε 2 .

Lemma 2.4. Let A := { i ∈ [ k ] | q i ⩾ α } . Let m 2 ⩾ 16384 max {√ 1 αε , √ k ε } be the number of samples used to compute Z 2 . Then E [ Z 2 ] = m 2 d χ 2 ( p A , q A ) . Moreover, if d χ 2 ( p A , q A ) ⩽ ε 2 , then Var[ Z 2 ] ⩽ ( 1 32 m 2 ε ) 2 . If d χ 2 ( p A , q A ) ⩾ ε , then Var[ Z 2 ] ⩽ O ( E [ Z 2 ] 2 ) .

Proof of Lemma 2.4. The proof is a relatively straightforward modification of the argument of [DKW18, Lemma 2]. We have the expectation and variance of Z 2 ,

It boils down to bounding the following,

Derivation of the inequalities follow from [DKW18, proof of Lemma 2].

Combing both, we have that

When d χ 2 ( p A , q A ) ⩽ ε/ 2 , then E [ Z 2 ] ⩽ m 2 ε 2 ; and we solve Var[ Z 2 ] ⩽ ( 1 32 m 2 ε ) 2 , which gives

We solve for the relaxation:

In the end, we obtain:

When d χ 2 ( p A , q A ) ⩾ ε , then E [ Z 2 ] ⩾ m 2 ε ; and we solve Var[ Z 2 ] ⩽ ( 1 4 E [ Z 2 ]) 2 ,

which is equivalent to the following

Further relaxing the solution, it is enough to have

as long as the following holds,

Letting m 2 ⩾ 512 max {√ 1 αε , √ k ε } , we have that both statements.

C Proofs of entropy testing lower bounds

Lemma 2.6. With fewer than c 3 · √ k/ε samples from p , no tester can distinguish between p = q and | H ( p ) -H ( q ) | ⩾ ε with probability higher than 2 / 3 , where c 3 > 0 is an absolute constant.

Proof of Lemma 2.6. This follows from the standard uniformity testing lower bound [Pan08], which provides a lower bound of Ω( √ k/η 2 ) : there exists a family of distributions that are hard to distinguish from uniform u k , using fewer than c 1 · √ k/η samples. Let k be an even number; the construction is by taking θ = {-1 , 1 } k/ 2 uniformly at random, and letting, for every i ∈ [ k/ 2] ,

We can verify that for any θ :

Setting η = ε 2 yields the lower bound of Ω ( √ k ε ) .

Lemma 2.7. With fewer than c 4 · log 2 k/ε 2 samples from p , no tester can distinguish between p = q and | H ( p ) -H ( q ) | ⩾ ε with probability higher than 2 / 3 , where c 4 > 0 is an absolute constant.

Proof of Lemma 2.7. Following [WY16, B.2 Proof of Proposition 2], we look at the same construction but with different parameters ε ′ = ε log(2( k -1)) :

One can check that

Moreover, direct calculation of the (squared) Hellinger distance shows that

which implies that p and q cannot be distinguished with fewer than c 4 log 2 k ε 2 samples [BY02, Theorem 4.7].

We will rely the following inequality for compression, both via Jensen's inequality,

as log( x ) is concave and log ( 1 x ) is convex.

suggesting that if we merge elements of ∆ into one, then we will lose a log( | ∆ | ) factor of the entropy. By merging enough elements, we can then reduce this problem into the first case, where elements have large enough mass in each location.

Claim D.1. Let S ⊆ [ k ] , if p ( S ) -q ( S ) > -η , then

Proof. If | d KL ( p S ∥ q S ) | ⩾ η , then

We will rule out the second case, using that log 1 x ⩾ 1 -x for x > 0 , 10

Thus, we cannot have d KL ( p S ∥ q S ) ⩽ -η and so d KL ( p S ∥ q S ) ⩾ η .

We use the same idea as our first upper bound but choose a series of thresholds. Let

The following calculation ensues

We have that one of the four terms will be at least Ω( ε/ 4) . If it is

which is testable with O ( log 2 ( k ) ε 2 ) samples using arguments from proof of Theorem 2.1. If it is | d KL ( p S i , q S i ) | ⩾ Ω( ε ) , for i = 1 , 2 , 3 . We have the following:

Case S 3 . Suppose | d KL ( p S 3 , q S 3 ) | ⩾ Ω( ε ) . We will check whether p ( S 3 ) ⩾ Ω ( ε log log log( k/ε ) ) , if not, we can reject. We proceed assuming the inequality holds. Note that

Thus, p ( S 3 ) -q ( S 3 ) > -O ( ε log log log( k/ε ) ) , and by Claim D.1, we have that d KL ( p S 3 , q S 3 ) ⩾ Ω( ε ) . Using (4), we then have that d χ 2 ( p S 3 , q S 3 ) ⩾ Ω( ε ) . Using Lemma 2.4 (setting α = O ( ε k log log log( k/ε ) ) ), and similar argument from the proof of Theorem 2.1, we have that O ( √ k log log log( k/ε ) ε ) suffices to check between the case that d χ 2 ( p S 3 , q S 3 ) ⩾ Ω( ε ) and p S 3 = q S 3 .

Case S 2 . Suppose | d KL ( p S 2 , q S 2 ) | ⩾ Ω( ε ) . We will check whether

if not, we will reject. We proceed assuming the inequality holds. Now, recall that the main bottleneck of the χ 2 tester analyzed in Lemma 2.4 is due to the minimum probability α = min i ∈S 2 q i (increasing this would decrease the sample complexity). Our main idea here is to increase α by merging a suitable number ( log log( k/ε ) in this case) of elements into one single bin to form a new distribution to test. Denote ∆ j where j ∈ [ |S 2 | log log( k/ε ) ] and ⋃ j ∆ j = S 2 . Wewill subsequently treat every elements in ∆ j as 1 bin in the new distribution, calling it p ∆ , q ∆ and denote p (∆ j ) , q (∆ j ) as mass on ∆ j , where p (∆ j ) = ∑ i ∈ ∆ j p i . This gives us the following:

iii. Their entropy difference is preserved, which we will prove next:

Note that these are better conditions compared to i. and ii. in the proof of Theorem 2.1 (in this analysis, using ii., it is sufficient to prove that d KL ( p ∆ , q ∆ ) ⩾ Ω( ε ) in view of Claim D.1). The gain comes from the fact that we can apply Lemma 2.4 with better α = min j q (∆ j ) ⩾ Ω ( ε k ) and thus

However, the gain only affect Claim 2.3 by constant factors. The soundness and completeness then follows similarly to the proof of Theorem 2.1. We prove (iii.) next:

Suppose H ( p S 2 ) -H ( q S 2 ) ⩾ ε , then,

where the (15) is due to (13) and for (16), recall that ∑ j p (∆ j ) = p ( S 2 ) ⩽ O ( ε log log log( k/ε ) ) . Suppose H ( q S 2 ) -H ( p S 2 ) ⩾ Ω( ε ) , the same goes below:

Therefore, we have proved (iii.).

Case S 1 . The proof follow similar to Case S 2 , but by merging log( k/ε ) elements.

E Proofs of testing Bayesian networks

Claim E.1. Suppose that p ( ¯ U ) ⩽ O ( ε 2 / log(1 /ε )) , then we have for any distribution q ,

This implies that d KL ( p ¯ U ∥ p G ; ¯ U ) ⩾ -O ( ε 2 ) for any DAG G . Moreover, if d KL ( p ∥ p G ) ⩽ O ( ε 2 ) , then d KL ( p U ∥ p G ; U ) ⩽ O ( ε 2 ) .

where we use monotonicity of -x log 1 x and the fact that p G ( ¯ U ) ⩽ 1 in the second last inequality. Since d KL ( p ∥ p G ) = d KL ( p U ∥ p G ; U ) + d KL ( p ¯ U ∥ p G ; ¯ U ) ⩽ O ( ε 2 ) and d KL ( p ¯ U ∥ p G ; ¯ U ) ⩾ -O ( ε 2 ) , we can rearrange and see that d KL ( p U ∥ p G ; U ) ⩽ O ( ε 2 ) .

We will need the following claim for the proof of Lemma 3.3.

Claim E.2. The following inequalities hold, for any i ∈ [ n ]

Proof. We will show (17) first.

where the first step follows from Jensen's inequality applied to the function f ( x ) = log(1 /x ) , and the second-to-last step is due to x log x is monotonically increasing when x ⩾ 1 e and log(1 -x ) ⩾ -2 x when x ∈ (0 , 0 . 5) . This concludes the proof of (17).

We next move on to proving (18).

̸

where the second-to-last step follows from Equation (19) below, and last step by monotonicity (decreasing) of f ( x ) = x log x when x ⩽ 1 e and ∑ x ∈A i \U p ( x ) ⩽ ∑ i = j ∑ x ∈A j p ( x ) ⩽ ∑ i = j O ( ε 2 n 2 log( n/ε ) ) ⩽ O ( ε 2 n log( n/ε ) ) .

̸

This concludes the proof of (18).

Lemma 3.3. Suppose d 2 H ( p, q ) ⩾ Ω( ε 2 ) ; d KL ( p ∥ p G ) ⩽ O ( ε 2 ) ; p ( ¯ U ) ⩽ ε 2 n log( n/ε ) ; ∀ i ∈ [ n ] , p ( ¯ A ′ i ) ⩽ ε 2 n 2 log( n/ε ) , where A ′ i is defined above, and q is Markov with respect to G , then we have

Therefore testing d KL ( p X i , Π G i ; A ′ i ∥ q X i , Π G i ; A ′ i ) ⩾ ε 2 n over all i suffices to detect this case.

Proof of Lemma 3.3. By Claim E.1 and the assumption that d KL ( p ∥ p G ) ⩽ O ( ε 2 ) , we have that

Ω( ε 2 ) ⩽ d 2 H ( p, q ) = d 2 H ( p U , q U )+ d 2 H ( p ¯ U , q ¯ U ) ⩽ d 2 H ( p U , q U )+ 1 2 ( p ( ¯ U )+ q ( ¯ U )) ⩽ d 2 H ( p U , q U )+ O ( ε 2 ) .

Ω( ε 2 ) ⩽ d 2 H ( p U , q U ) ⩽ d KL ( p U ∥ q U ) + q ( U ) -p ( U ) ⇒ d KL ( p U ∥ q U ) ⩾ Ω( ε 2 ) . (21) Combining and (20) and (21), we write

With Claim E.2 and Claim E.1, we can lower bound the sum of local KL divergences,

Lemma 3.4. Let p and q be two max in-degreed Bayes nets supported on { 0 , 1 } n such that for every subset L ⊆ { X 1 , · · · , X n } of size d +1 , the following holds:

Suppose p is Markov w.r.t. G ′ and q Markov w.r.t. G . Then we have that

Proof of Lemma 3.4. More formally, by the celebrated works of Chow and Liu [CL68] and its generalization to Bayes net, one can write the KL divergence between a Bayes net and its projection to any graph G as difference between sum of n local conditional entropies (we provide a derivation for the sake of completeness in Appendix F):

where X Π i denotes the parents of X i in Bayes net p and X Π G i denotes the parents of X i in DAG G . Here we assume that q is Markov with respect to G . Since the local entropies between p and q are close by O ( ε 2 /n ) (see (6)) and its relation to conditional entropy via (3), we can conclude the following:

Since we assume that q is Markov with respect to G , we can combine (22), (23), (24) and (25), which then give:

where p is Markov with respect to G ′ , as Π i is the set of parents of X i for p . Rearranging terms and we have

Since KL -divergence is nonnegative, we conclude that both terms must be at most O ( ε 2 ) .

F Derivation for KL decomposition

Below, we provide a full proof on decomposing the KL divergence between a Bayes net p and its projection p G , for any DAG G , into local conditional entropies.

where π i , Π i denote the parents of x i , X i in Bayes net p (a set of random variables or their domain); and π G i , Π G i as the parents defined by G . p G is the projection of p unto G as defined by Definition 3.2. It is not hard to see that the derivation extends beyond the case of hypercube, { 0 , 1 } n .

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proofs are provided either in the main paper or the appendix, both part of the submission. Assumptions are fully stated in the theorem and lemma statements.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper does not contain experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: this work is theoretical in nature; it is hard to predict its societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors first find the complexity upper and lower bound for the entropy identity testing problem: given sample access to a distribution $p$ and a fully described distribution $q$, the tester needs do distinguish between two hypotheses $p=q$ and $|H(p)-H)(q)|\\geq \\epsilon$. Based on this, the authors find the sample complexity upper bound for the problem of identity testing for in-degree-$d$ $n$-dimensional Bayesian networks. This bound improves an existing bound by Canonne et al. (2020) without using an additional assumption that the structure of unknown Bayes net is a subset of that of the reference one.\n\nSTRENGTHS:\nThe claimed results sound interesting.\n\nWEAKNESSES:\nSome key proofs look not correct, and some notations are not defined. Please see the questions below.\n\nQUESTIONS:\n+ What is the definition of $\\mbox{Poi}(m)$ in Claim 2.3?\n+ Proof of Theorem 2.1: The first inequality between lines 195 and 196 looks not correct since the function $\\log \\frac{1}{x}$ is convex. An application of the Jensen's inequality should give $H(q_{\\bar{A}}) \\geq q(\\bar{A}) \\log \\frac{|\\bar{A}|}{q(\\bar{A})}$ (not $\\leq$ as in your paper). \n+ For the case $p=q$, from the line 206, we only infer that $Z_2<2m_2 \\epsilon$ with probability $>3/4$. What happens in Algorithm 1 if $2m_2 \\epsilon> Z_2\\geq \\frac{1}{16}m_2 \\epsilon$?\n+  In the last equations in Appendix A (Derivation of Line 4 in Algorithm 1), you mentioned that you use Chernoff bound $\\mathbf{P}(\\hat{p}'(\\bar{A})\\leq (1-1/3)p(\\bar{A}))\\leq \\exp\\big(-\\frac{m_1 p(\\bar{A})}{18}\\big)$. Would you please explain this more? It seems to me that you need use the union bound before using the Chernoff bound, hence we need to multiply $|\\bar{A}|$ in the RHS of the above inequality.\n+ In the statement of Lemma 2.4, $\\mbox{Var}[Z_2]\\leq O(m_2^2 \\epsilon^2)$ and $\\mbox{Var}[Z_2] \\leq O(\\mathbf{E}[Z_2]^2)$. However, in the proof (from line 204 to line 223), you use the exact constants. \n+ In line 218, you mention that you use Claim 2.3 to derive some results. How do you justify that $d_{\\chi^2}(p_A,q_A)\\leq \\epsilon/8$ to apply this claim?\n\nLIMITATIONS:\nThis is a theoretical research paper, hence the negative society impact of this work is not direct.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the entropy identity testing problem for two discrete distributions $p$ and $q$, which is a hypothesis test between $p=q$ and $|H(p) - H(q)| \\geq \\epsilon$. They propose an algorithm for this problem that is near-optimal in terms of the sample complexity. The main ideas of the algorithm are 1) ignoring a set with small probability with respect to $q$ and 2) bounding the entropy difference by KL-divergence. The result is applied to identity testing problem for Bayes nets.\n\nSTRENGTHS:\nThe performance of the algorithms, including the runtime and the error bound, is guaranteed by theoretical analysis.\nDetailed mathematical proof is given.\n\nWEAKNESSES:\nThe importance of the problem is not discussed much.\nThe writing is not clear with several errors.\n\nQUESTIONS:\n1) It is hard to see the significance of the entropy identity testing. \nHeuristically, the alternative hypothesis $|H(p) - H(q)| > \\epsilon$ is too much different from the null, since it is possible that $p \\neq q$ but $H(p) =H(q)$,\nand thus the entropy identity testing might be essentially easier than the other testing problems based on \"distances\" such as KL-divergence or Hellinger distance.\n\n2) I think the manuscript should be revised for clarity, since it contains too many typos and small errors. Below are a few examples.\n- In Abstract, $k$ is not defined.\n- In line 131, what is $\\leftarrow$? Also, if $\\bar{A}$ is the complement of $A$, then the equality should be removed from $q_i \\leq \\tau/k$.\n- In the equation below line 134, $p(\\bar{A}) = O(\\tau)$ does not imply the second inequality.\n- In line 182, the semicolon seems to be a typo.\n- In Algorithm 1, what is $N_i$?\n- In line 200 and line 202, \"Algorithm 4\" should be \"Algorithm 1\".\n- In Algorithm 2, what is S_1?\n\nLIMITATIONS:\nThe work does not seem to have potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n* This work focuses on the problem of Entropy identity testing, i.e., whether for two distributions $p, q$ if $p = q$ or $|H(p) - H(q)| > \\epsilon $ given samples from unknown  $p$ and complete description of $q$ over a domain of size $k$.\n\n* The authors show that the sample complexity of entropy identity testing is $ \\sqrt{k \\log(k/\\epsilon )} /\\epsilon + \\log^2(k)/\\epsilon^2 $. This is an interesting observation since for identity testing problems, the sample complexity is usually $O(\\sqrt{k}/\\epsilon^2)$, but now it gets split into two terms with $\\epsilon^2$ appearing below the shorter term when $k$ is exponentially large. \n\n* Using this testing procedure as a subroutine, this results leads to better testing procedure for the problem of Identity testing for Bayesian networks over {0,1}$^n$ and the authors (as mentioned in abstract) obtain a better bound for this problem improving it from $O(2^{d/2} n^2 /\\epsilon^4 )$ in [CDKS20] to $O(2^{d/2} n \\epsilon^2 + n^2/\\epsilon^4)$.  \n\nRoughly speaking, the algorithm for entropy identity testing divides the space into two regions:\n1) Where $q$ has low probability: just test whether $\\hat{p}$ attains large values in this region. if so, then reject.\n2) Where $q$ has high probability: use the algorithm of [DKW18] to efficiently test whether $d_{KL}(p||q) \\geq \\epsilon$.\n\nSTRENGTHS:\n* The nice thing is the observation that the identity entropy testing problem is much easier than the entropy estimation problem itself, which is known to be $\\theta( \\frac{k}{\\epsilon \\log k}  + \\frac{\\log^2 k}{\\epsilon^2} )$.\n\n* The paper is well explained. I appreciate the intuitive explanation and thought process of bounds that are obtained using simple ideas, followed by the need to make appropriate modifications to overcome those issues.\n\n* It is indeed surprising that entropy, which is not an easy quantity to estimate (optimally), is much easier to test.\n\nWEAKNESSES:\n* The algorithm/testing-procedure, although interesting from a theoretical viewpoint, is hardly practical as it involves so many individual tests and just too many weird (but tunable) constants. A \"neat algorithm\" would have been even more useful from a practical viewpoint, but overall the work is interesting and may serve as a nice starting point for designing a ``neat\" algorithm. \n\nOther minor points: \n* The (possible) applications of either of the two testing problems, especially the identity testing of degree-d Bayes can be mentioned.\n\n* Lines 142-143 can be slightly rephrased ``directly extending this argument does not work\" causes some confusion.\n\n* In line 145, please provide a reference/explanation for where the factor $\\epsilon / \\sqrt{n}$ comes from. If I am not wrong, it probably comes from [DP16 Theorem 4.2]\n\nQUESTIONS:\nAs far as the goal of getting a better bound for identity testing for Bayes net is concerned, won't Reyni Entropy identity testing might be a better choice (especially of order $2$), which has $\\sqrt{k}$ sample complexity and is also much simpler to estimate and might have even better sample complexity for identity testing.\n\nLIMITATIONS:\nWork has essentially no negative societal impact. A justification for the limitations question in the checklist can be added as it could help the readers directly without searching at different places.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of testing a null hypothesis against alternatives that are far in Shannon entropy. They provide information-theoretic minimax lower bounds that they also achieve up to polylogarithmic factors. They then apply these results to Bayes net testing.\n\nSTRENGTHS:\n* Clear and well-written\n* This seems like a fundamental problem and it’s good to get tight bounds for it\n\nWEAKNESSES:\n* The lemmas in section 2.2 are written somewhat informally, but they read as if they are making an instance-dependent claim (e.g., for any $q$, it is impossible to test with fewer than $c_3 \\sqrt{k}/\\varepsilon$). However, the lemmas seem to actually be minimax claims, as the two lower bounds are obtained by different single hard distributions.\n* There is no discussion of proof strategy or intuition in section 2.2 so the results feel like they come from no where (the results of lemma 2.6 were discussed in section 1.2, but not lemma 2.7 as far as I can tell).\n* The fact that the two lower bounds arise from such different distributions indicates the possibility of a tighter instance-dependent quantity. This sort of bound could help illuminate the structure of this testing problem.\n\nQUESTIONS:\n* In the display after line 195, I don’t quite follow how the last inequality holds. In particular, where does the $\\log \\log (k/\\varepsilon)$ factor go?\n* Between equations (13) and (14) in the display following line 416, $d_{TV}(p, q)$ is bounded above by $\\sqrt{\\frac{\\varepsilon}{8}} + 4\\tau$, which is then bounded above by a constant. Isn’t it easier to just directly (and trivially) bound $d_{TV}(p, q)$ by 1? This would also remove the requirement (as far as I can tell) of assuming $p(\\bar{A}) + q(\\bar{A}) \\leq 4\\tau$.\n\nMinor comments\n* In line 108, in the phrase “as just mentioned, when its TV distance…” what is “it” referring to?\n* In line 163, is the partial Hellinger distance missing a power of 2 in the summand?\n* In lines 414-416, the sentence “We continue based on the premise…” is redundant, as the premises given are already assumed as conditions in the claim.\n* In the display following line 416, maybe good to mention that the Poissonization trick is used there\n* In line 434, “When $d_{\\chi^2}(p_{\\mathcal{A}}, q_{\\mathcal{A}}) \\geq \\varepsilon$” is written twice.\n* In the display following line 446, should $\\theta \\cdot \\eta$ instead be $\\theta_i \\cdot \\eta$?Otherwise I don’t see how the equation can make sense ($\\theta$ is a vector, $\\eta$ is a scalar, and the output of the operation needs to be a scalar).\n\nLIMITATIONS:\nThey have addressed their limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}