Title: Low Degree Hardness for Broadcasting on Trees
Venue: NeurIPS 2024
Accepted: yes
Keywords: Broadcasting Process

low degree hardness

statistically-computational gap
Abstract: We study the low-degree hardness of broadcasting on trees.
Broadcasting on trees has been extensively studied in statistical physics, 
in computational biology in relation to phylogenetic reconstruction and in statistics and computer science in the context of block model inference,  and as a simple data model for algorithms that may require depth for inference. 

The inference of the root can be carried by celebrated Belief Propagation (BP) algorithm which achieves Bayes-optimal performance. Despite the fact that this algorithm runs in linear time (using real operations), recent works indicated that this algorithm in fact requires high level of complexity. 
Moitra, Mossel and Sandon constructed a chain for which estimating the root better than random (for a typical input) is $NC1$ complete. Kohler and Mossel constructed chains such that for trees with $N$ leaves, recovering the root better than random requires a polynomial of degree $N^{\Omega(1)}$. Both works above asked if such complexity bounds hold in general below the celebrated {\em Kesten-Stigum} bound. 

In this work, we prove that this is indeed the case for low degree polynomials. 
We show that for the broadcast problem using any Markov chain on trees with $N$ leaves, below the Kesten Stigum bound, any $O(\log N)$ degree polynomial has vanishing correlation with the root.  

Our result is one of the first low-degree lower bound that is proved in a setting that is not based or easily reduced to a product measure.

Low Degree Hardness for Broadcasting on Trees

Abstract

We study the low-degree hardness of broadcasting on trees. Broadcasting on trees has been extensively studied in statistical physics, in computational biology in relation to phylogenetic reconstruction and in statistics and computer science in the context of block model inference, and as a simple data model for algorithms that may require depth for inference.

The inference of the root can be carried by celebrated Belief Propagation (BP) algorithm which achieves Bayes-optimal performance. Recent works indicated that this algorithm in fact requires high level of complexity. Moitra, Mossel and Sandon constructed a chain for which estimating the root better than random (for a typical input) is NC 1 complete. Kohler and Mossel constructed chains such that for trees with N leaves, recovering the root better than random requires a polynomial of degree N Ω(1) . Both works above asked if such complexity bounds hold in general below the celebrated Kesten-Stigum bound.

In this work, we prove that this is indeed the case for low degree polynomials. We show that for the broadcast problem using any Markov chain on trees with N leaves, below the Kesten Stigum bound, any O (log N ) degree polynomial has vanishing correlation with the root.

Our result is one of the first low-degree lower bound that is proved in a setting that is not based or easily reduced to a product measure.

1 Introduction

Understanding the computational complexity inference problems of random instances has been extensively studies in different research areas including statistics, cryptography, computational complexity, computational learning theory and statistical physics. The emerging field of research is mainly devoted to the study of computational-to-statistical gaps. ([3, 41, 24])

Recently, low-degree polynomials have emerged as a popular tool for predicting computationalto-statistical gaps, especially in the context of the Bayesian framework. Our work follows [23] in studying the polynomial hardness of broadcasting on trees.

A very exciting line of work, including [19, 18, 25, 4, 15, 26, 17, 9, 40] recently showed that the 'low-degree heuristic' can be used to predict computational-statistical gaps for a variety of problems such as recovery in general stochastic block models, sparse PCA, tensor PCA, the planted clique problem, certification in the zero-temperature Sherrington-Kirkpatrick model, the planted sparse vector problem, and for finding solutions in random k -SAT problems.

Interestingly, it was observed that the predictions from this method often agree with predictions from statistical physics heuristics based on the replica and cavity methods which are closely related to the analysis of BP/AMP fixed points, see e.g. [12, 13, 28]).

It is often argued that low-degree polynomials algorithms are relatively easy to use (e.g. compared to proving SOS lower bounds), and that low degree polynomials capture the power of the 'local algorithms' framework used in e.g. [16, 10] as well as algorithms which incorporate global information, such as spectral methods or a constant number of iterations of Approximate Message Passing [29].

In this work, we continue to study the power of low-degree polynomials for the (average case) broadcast on trees problem. In broadcasting on trees the goal is to estimate the value of the Markov process at the root given its value at the leaves and the goal is to do so for arbitrarily deep trees. Two key parameters of the model are the arity of the tree d and the magnitude of the second eigenvalue λ of the broadcast chain.

A fundamental result in this area [22] is that when d | λ | 2 > 1 nontrivial reconstruction of the root is possible by counting the number of the leaves of different types, an algorithm that could be described as a linear function of the leave values. In contrast, when d | λ | 2 < 1 , such linear estimators have no mutual information with the root (but more complex statistics of the leaves may) [30, 36].

This threshold d | λ | 2 = 1 is known as the Kesten-Stigum threshold . A series of works showed that the KS threshold is the information theory threshold for non-trivial root inference for some specific channels, including the binary symmetric channel [6, 14, 21, 20] and binary channels that are close to symmetric [8], as well as 3 × 3 symmetric channels for large d [39].

While the Kesten-Stigum bound is easy to compute, it turns out that in many cases, it is not the information-theoretic threshold for root recovery. This was first established in [30] for symmetric channels with sufficiently many states, specifically when q ≥ C for some large constant C . Later it was shown for symmetric channels with q ≥ 5 states in [39]. Recently, the results [37] provide more information about the case of q = 3 and q = 4 . Many of the finer results in this area prove predictions from statistical physics. The connection between the broadcast problems and phase transitions in statistical physics was made in [27], and more recent predictions include [5, 2, 38]. Moreover, the information-theoretic threshold may depend on the specific structure of the channel rather than solely on d and λ . Notably, [30] also showed that there exists channels where non-trivial predictions of the root are achievable even when | λ | = 0 .

Much of the interest in Kesten-Stigum threshold comes from the fundamental role it plays in problems, such as algorithmic recovery in the stochastic block model [11, 33, 7, 35, 1] and phylogenetic reconstruction [31]. Count statistics can be viewed as degree 1 polynomials of the leaves, which begs the question of what information more general polynomials can extract from the leaves. See [32, 34] for surveys on the topic.

In [23] it was shown that λ = 0 even polynomials of degree N c , where N = d ℓ is the number of leaves of for a d -ary tree of depth ℓ , for a small c > 0 are not able to correlate with the root label (as ℓ tends to ∞ ), whereas computationally efficient reconstruction is generally possible as long as d is a sufficiently large constant [30].

The main motivation of [23] was to prove that low degree polynomials fail below the Kesten Stigum bound: 'It is natural to wonder if the Kesten-Stigum threshold d | λ | 2 = 1 is sharp for low-degree polynomial reconstruction, analogous to how it is sharp for robust reconstruction." However the main result of [23] only established this in the very special case of λ = 0 . This problem is also stated in the ICM 2022 paper and talk on the broadcast process [34]: ' The authors of [23] ask if a similar phenomenon holds through the non-linear regime. For example, is it true that polynomials of bounded degree have vanishing correlation with X 0 in the regime where dλ 2 < 1 ? " The main results of this paper prove that this is indeed the case. We proceed with formal definitions and statement of the main result.

1.1 Definitions and Main Result

Rooted Tree

Recall that every rooted tree T inherently defines a partial order relation among its vertices: For a pair of distinct vertices, u is said to be an ancestor of v (and v a descendant of u ), denoted as v < u in this paper, if u is contained in the unique path from v to the root ρ . Specifically, if v and u are directly connected by an edge, we also refer to v as a child vertex of u (and u as the parent vertex of v ). By v ≤ u we mean that v is either a descendant of u or v = u . In general, if v < u and the path distance between them is k , v is referred to as a k th-descendant of u (and u as the k th ancestor of v ).

For a nonnegative integer k , the k th layer of the tree refers to the set of k th descendants of the root ρ . (The root here is considered at the 0th layer of the tree.) The depth of the tree is defined as the largest non-negative integer ℓ for which the ℓ th layer is not empty, and we denote the ℓ th layer by L .

The height of a vertex u is defined as

In particular, when L is the set of leaves, then h( u ) is simply the distance from u to L . In this paper, we may abuse the notation by writing x ∈ T or S ⊆ T to mean that x is a vertex or S is a subset of vertices in the tree T .

The standard rooted d -ary tree with depth ℓ is a tree where each vertex u / ∈ L has exactly d children vertices. Let us start by defining the type of trees we will be investigating in this paper, which is a slight generalization of d -ary tree.

p

t

m

u

S

Definition 1.1. A rooted tree T with root ρ has degree dominated by d ≥ 1 with parameter R ≥ 1 if for every vertex u and positive integer k , the number of k th descendants of u is at most Rd k .

With the above definition, a d -ary rooted tree has degree dominated by d ≥ 1 with parameter R = 1 . Further, a typical realization of a Galton-Watson tree of Poisson type with average degree d and of depth ℓ (a random tree in which each vertex u / ∈ L has on average d children vertices) has a degree dominated by d ≥ 1 with parameter R ≃ log( ℓ ) .

Broadcasting Process on Rooted Trees

Next, we will define the broadcasting process on a rooted tree T with root ρ . Consider a q × q ergodic transition matrix M , where q ≥ 2 . Recall that every eigenvalue of a transition matrix M has an absolute value at most 1. Let 0 ≤ λ ≤ 1 represent the second largest absolute value among the eigenvalues of M . Additionally, we define the stationary distribution of M as π .

The broadcasting process X = ( X v ) v ∈ T , with state space [ q ] := { 1 , 2 , . . . , q } and transition matrix M , can be formally described as follows: We initialize the value of X ρ according to some initial distribution ν . As we reveal the values layer by layer, when the value X u is revealed, the value X v for any child vertex v of u is independently distributed according to a specific row of M depending on the value of X u :

In other words, each vertex's value depends only on its parent vertex's value. The definition of the process is given below:

Definition 1.2 (Broadcasting Process on Tree) . Let q ≥ 2 be a positive integer. For any rooted tree T with root ρ and a q × q ergodic transition matrix M , the broadcasting process X = ( X v ) v ∈ T with state space [ q ] , according to transition matrix M with an initial distribution ν for X ρ , is a random process with joint distribution given by:

where the product is taken over all pairs ( v, u ) such that v is a child vertex of u .

For a subset of vertices A ⊂ T , let us denote

If the tree is just a path, then the process reduces to a Markov chain. If we assume ν = π , then X v ∼ π for every v ∈ T , as ( X v ) v ∈ P for every downward path of T forms a Markov Chain with transition matrix M . Further, let us make a remark about the Markov property of the process.

Remark 1.3 (Markov Property) . The broadcasting process establishes a Markov Random Field on tree T : Given any three disjoint subsets A,B, and C of T , if every path from a vertex in A to a vertex in C passes through a vertex in B , then the random variables X A and X C are conditionally independent given X B .

Polynomials of x L and the Main Result

Definition 1.4. Let x ∈ [ q ] T . For u ∈ T , let x ≤ u = ( x v ) v ≤ u . For subset U ⊆ T , let x U = ( x u ) u ∈ U .

The next definition is about the notion of degrees for functions with variables x L = ( x v ) v ∈ L . This is the generalization of degree of a polynomial.

Definition 1.5 (Efron-Stein Degree) . A function f with variables x L has Efron-Stein degree at most k if it can be expressed as

where the summation is over a finite set of indices α , and each f α is a function of the variable x S for some S ⊆ L with | S | ≤ k .

Now, we could properly formulate the main result of the paper:

Theorem 1.6. Let T be a rooted tree with root ρ of depth ℓ and has degree dominated by d ≥ 1 with parameter R ≥ 1 . Consider the broadcasting process on T with a q × q transition matrix M and X ρ ∼ π . If M is ergodic and dλ 2 < 1 , then there exists a constant c > 0 which depends on M and dλ 2 such that the following holds: For any function f ( x L ) of Efron-Stein degree ≤ c ℓ 1+log( R ) , we have

Remark 1.7. Given that dλ 2 < 1 implies λ 2 < 1 , and λ < 1 if and only if λ 2 < 1 , we can infer that λ < 1 from the given conditions. Consequently, the term max { dλ 2 , λ } < 1 follows from the assumptions of the theorem. Therefore, the R.H.S. of the inequality decays exponentially with the depth of the tree.

Follows from the theorem and propteries of conditonal expectation, we have the following corollary.

Corollary 1.8. With the same setting as in Theorem 1.6, for any function f ( x L ) of Efron-Stein degree ≤ c ℓ 1+log( R ) , and any function g ( x ρ ) of the root value, their correlation satisfies

The proof of the theorem is based on recursion on a notion of fractal capacity of functions. Indeed, the main result is optimal in the fractal sense (Theorem 1.16), as we will later demonstrate that all functions with fractal capacity up to a level proportional to ℓ exhibit vanishing correlation with the root, whereas all functions of the leaves have fractal capacity at most ℓ +1 . Let us introduce the necessary definitions and notations to introduce both the fractal capacity and the proof overview.

1.2 Fractal Capacity and Proof Overview

To provide a clearer illustration, we establish a correspondence between the vertices of T and words of varying lengths from 0 to ℓ , with vertices at the k th layer represented as words of length k . We

denote the root ρ as the empty word () . For each vertex u , represented by the word ( b 1 , b 2 , . . . , b k ) , we define d u as the number of children vertices of u , and we identify these children vertices as ( b 1 , b 2 , . . . , b k , i ) with i ∈ [ d u ] := { 1 , 2 , . . . , d u } . Notice that v is a descendant of u is equivalent to u is a prefix of v . For brevity, for each u = ( b 1 , . . . , b k ) ∈ T and i ∈ [ d u ] , let

Furthermore, we denote the parent vertex of u as p ( u ) = ( b 1 , . . . , b k -1 ) and the set of children vertices of u as c ( u ) = u [ d u ] .

(1)

(2)

u

(11)

(12)

()

(21)

u

(111)

(121)

(122) (131)

(132)

(211)

(212)

u

u

u

21

22

32

Definition 1.9. For a non-empty subset S ⊆ L , we introduce the notation ρ ( S ) to represent the nearest common ancestor of the vertices in S , meaning that ρ ( S ) is the vertex with smallest height that is an ancestor of all vertices in S .

Here we consider the case when | S | > 1 . Notice that ρ ( S ) is not at the ℓ th layer L . For each child ρ ( S ) i of ρ ( S ) for i ∈ [ d ρ ( S ) ] , we define the set S i as

which is the collection of vertices in S that are descendants of ρ ( S ) i . Let

̸

Then, S can be expressed as the disjoint union of the sets S i for i ∈ I ( S ) :

We call the above disjoint union the branch decomposition of S , and each S i for i ∈ I ( S ) a branch part of S .

The branch decomposition is a key concept in the proof, and we define the fractal capacity according to the number of iterations to decompose S into singletons.

Definition 1.10. Let

be the collection of singletons of L . We say a subcollection A ⊆ 2 L \{∅} is closed under decomposition with base A 1 if for every S ∈ A\A 1 , we have S i ∈ A for i ∈ I ( S ) .

Definition 1.11. For any A ⊆ 2 L \{∅} , let

be a new subcollection defined according to the following rules:

For any S ∈ 2 L \{∅} , S ∈ B if and only if one of the following two conditions holds

For I ⊆ [ d u ] , let

S ∈ A 1 .

S / ∈ A 1 and S i ∈ A for i ∈ I ( S ) .

For example, B ( A ) contains sets of size ≤ 2 .

Lemma 1.12. If A 1 ⊆ A ⊆ 2 L \{∅} is a subcollection closed under decomposition with base A 1 , then the collection B = B ( A ) contains A and it is also closed under decomposition with base A 1 .

Proof. To show A ⊆ B , it is sufficient to show A\A 1 ⊆ B . For any S ∈ A\A 1 , because A is closed under decomposition, S i ∈ A for i ∈ I ( S ) . Hence, S ∈ B follows from the definition of B . Now, for S ∈ B\A 1 , each S i with i ∈ I ( S ) is contained in A ⊆ B , which in turn implies B is closed under decomposition.

Now, we define recursively that

for positive integer k ≥ 2 . Observe the following two facts: Consider any non-singleton set S ⊆ L and S i with i ∈ I ( S ) . First, S ∈ A k ⇒ S i ∈ A k -1 by the definition of A k . Second, ρ ( S ) > ρ ( S i ) by the definition of branch decomposition. Given these two facts, we can prove inductively that

Since that there are only ℓ layers of the tree, we conclude that every non-emptyset of S ⊆ L is in A ℓ +1 . Therefore, together with Lemma 1.12, we have the following chain of subcollections:

Definition 1.13 (Fractal Capacity) . For any non-empty subset S ⊆ L , we define the fractal capacity of S as the smallest k such that S ∈ A k .

We introduce the notion of fractal capacity, borrowing terminology from fractal geometry. The recursive nature of our definition on subsets of trees mirrors the self-similar complexity found in fractal structures. This recursive and inherently intricate structure motivates our choice of the term fractal capacity, capturing the fractal-like properties that emerge in the collections ( A k ) k ∈ [ l +1] .

Definition 1.14. Given a collection A ⊆ 2 L \{∅} . A function f : [ q ] T → R is called an A -polynomial if we can express

where each f S is a function of x S = ( x v ) v ∈ S . A function f : [ q ] T → R has fractal capacity ≤ k if it is a A k -polynomial.

Remark 1.15. It is not hard to verify that A k contains all non-empty subsets S ⊆ L with | S | ≤ k . Thus, for any function f with variables x L ,

Efron-Stein degree of f ≤ k ⇒ f is an A k -polynomial .

On the other hand, it is worth to remark that for the d -ary tree of depth ℓ ≥ k , there exists S ∈ A k with | S | = d k -1 . (Namely, taking S = { v ∈ L : v < u } for some u with h( u ) = k -1 .) Thus, an A k -polynomial could have an Efron-Stein degree exponential in k .

The main result of the paper in terms of the fractal capacity is the following:

Theorem 1.16. With the same setting as in Theorem 1.6, there exists a constant c > 0 which depends on M and dλ 2 such that the following holds: For any function f ( x L ) with fractal capacity ≤ c ℓ 1+log( R ) , we have

Indeed, Theorem 1.6 is a direct consequence of Theorem 1.16, as A k -polynomials contains all polynomials of Efron-Stein degree ≤ k . Further, in terms of fractal capacity, the theorem is optimal because the correlation decay persists up to an order proportional to ℓ , while a fractal capacity ≤ ℓ includes all functions of the leaves.

Overview of the Proof Idea: For illustration, let us consider the case where T is a binary tree of depth ℓ with M = [ 1+ λ 2 1 -λ 2 1 -λ 2 1+ λ 2 ] , such matrix has eigenvalues λ and 1 . Here we assume 2 λ 2 < 1 .

We recall that for this binary symmetric broadcasting process, it is information-theoretically impossible to recover the root label from the leaves below the KS bound. This implies that all polynomials of X L have vanishing correlation with X ρ . Still, we use this simple process to illustrate the proof idea as our arguments for low-degree polynomials generalize to general broadcasting processes below the Kesten-Stigum threshold, including cases where it is information theoretically possible to estimate the root from the leaves non-trivially (in such cases there exist functions f so that lim inf ℓ →∞ Corr( f ( X L ) , X ρ ) > 0 ).

Now, let us consider degree1 polynomials. Suppose f is a A 1 -polynomial (equivalently, of EfronStein degree 1), we can express it in the form

where each f u is a function of x u . Given our focus on the variance, we may assume E f u ( X u ) = 0 for each u ∈ L . Then, our goal is to prove E [( E [ f ( X L ) | X ρ ] ) 2 ] is negligible comparing to E [ ( f ( X L )) 2 ] . Following from the Cauchy-Schwarz inequality that

we have

where the second inequality is derived from the variance decay property of in a Markov Chain. Thus, if we can establish ∑ u ∈ L E [ ( f u ( X u )) 2 ] is at the same order as E [ ( f ( X L )) 2 ] , the proof is complete. Notice that

̸

Thus, the goal here is to show

̸

for some constant c ∈ (0 , 1) , which in turn implies the desired result:

Roughly speaking, (4) holds if for most pairs u and v within L , the correlation between f u ( X u ) and f v ( X v ) is sufficiently small, which is the case for degree-1 polynomials.

Now, let us take a closer look. Fix any two vertices u and v in L , with w as their nearest common ancestor. Suppose u ≤ w 1 and v ≤ w 2 . Let X ̸≤ w 1 = ( X u ′ ) u ′ ̸≤ w 1 . We have

where the last inequality follows from the variance decay of the Markov Chain for length h( w ) . The above inequality implies that the correlation between f u ( X u ) and f v ( X v ) is at most of order λ h( w ) .

The above bound can be improved to λ 2 k by also taking the conditional expectation E [ f v ( X v ) | X ̸≤ w 2 ] into account, which requires the Markov Property that X u and X v are independent conditioned on X w . From here, one can properly arrange the terms and apply Cauchy-Schwarz inequality to show (4) holds.

Our proof of the main theorem tries to generalize the argument above to low degree polynomials. Let us summarize it by the following five pieces of descriptions.

I. Bounding Covariance: First, we generalized the idea on how (5) works for degree-1 polynomials. Suppose f α and f β are two functions so that the following holds.

f α ( x S ) is a function of x S for some set S ⊆ L such that

where we use a ≪ b to indicate a is much smaller than b . We keep this not precise to avoid technical details, but expect that the ratio is exponentially small in h( w ′ ) .

f β ( x S ′ ) is a function of x S ′ with S ′ ⊆ L satisfying S ′ ∩ { v ′ : v ′ ≤ w ′ } = ∅ .

Then, following the same derivation as shown in (5) we have

(See Figure 3 for an illustration.)

w

II. Choosing a good decomposition of the function: In essence, our proof strategy for any given function f ( x L ) with E f ( X L ) = 0 revolves around decomposing f ( x L ) into a sum of functions f α ( x L ) for α in some index set I , such that

|I| ≲ 2 ℓ ,

For each α , E f α ( X L ) = 0 and E [( E [ f α ( X L ) | X ρ ] 2 ] ≪ E [( f α ( X L )) 2 ] ,

̸

Whenever α = β , we can find w ∈ T so that f α and f β satisfy the covariance bound in I . (Possibly with a switch of the roles of α and β ).

Let us remark that while we are writing f α ( x L ) , it does not mean that f α depends on every leave variables.

If this is the case, then we could follow the argument in the degree 1 case to show that desired result holds.

III. From A k polynomials to A k +1 polynomials: The proof of the main theorem builds on I and II and advancing through a recursion on the fractal capacity of the function. This recursive approach relies on the following property:

Suppose we have shown the second moment decay A k -polynomials with mean 0 in the following sense: For every A k -polynomial f ( x S ) with mean 0 and variable x S where S ⊆ { v ∈ L : v ≤ ρ ′ } for some vertex ρ ′ ,

where h A k is some penalty constant depending on A k . (The bigger the value h A k , the weaker the second moment decay.)

Consider a specific type of A k +1 -polynomials. Fix a pivot vertex w with h( w ) large enough, let x ≤ w i = ( x v ) v ≤ w i for i ∈ [2] . Let g be a function of the form

where J is a finite index set and every g α,i is an A k -polynomial whose variables are x ≤ w i (more precisely, its variables are x S ′ where S ′ ⊆ { v ∈ L : v ≤ w i } ) and E g α,i ( X ≤ w i ) = 0 . Then, the function g ( x ) is a A k +1 -polynomial with variable x S for S ⊆ { v ∈ L : v ≤ w } .

Observe that, if we fix x ≤ w 2 , then x ≤ w 1 ↦→ g ( x ≤ w 1 , x ≤ w 2 ) is an A k -polynomial with variable x S ′ for S ′ ⊆ { v ∈ L : v ≤ w 1 } and mean 0 . This allows us to apply the assumption for A k polynomials to show

Clearly, the same inequality holds with the roles of x ≤ w 1 and x ≤ w 2 switched.

Base on this, one key step in the paper is to show g satisfies

This inequality is immediate if X ≤ w 1 and X ≤ w 2 are independent, which is not the case in the broadcasting process. One of the main technical challenges in the proof is to show that the inequality holds when X ≤ w 1 and X ≤ w 2 are conditionally independent given X w by the Markov Property. It turns out to impose a significant technical challenge when some entries of M can be 0 .

Observe that (7) not only implies g satisfies the desired second moment decay for II (2) , but also I (1) with w ′ to be either w 1 or w 2 . Indeed, these two properties will also hold for ˜ g ( x ) := g ( x ) -E g ( X ) , the normalized g with mean 0 , due to ( E g ( X )) 2 is negligible comparing to E g ( X ) 2 .

IV. A closer look at the decomposition:

Consider any A k +1 -polynomial f of variable x S for S ⊆ { v ∈ L : v ≤ ρ ′ } for some vertex ρ ′ . Before we proceed to the discussion of the decomposition, we remark that one cannot simply express f in the form of ˜ g described above with any pivot vertex w .

Let us give an example to illustrate why: Consider two functions f 1 , f 2 , where f i is a function of x S ′ with S ′ ⊆ { v ∈ L : v ≤ ρ ′ i } and S ′ ∈ A k +1 \A k . Let f = f 1 + f 2 . Then one can justify that f cannot be expressed in the form of ˜ g with any pivot w by using the property "if we fix x ≤ w 2 , then x x ≤ w 1 ↦→ g ( x ≤ w 1 , x ≤ w 2 ) is a A k -polynomial" discucssed in III to derive a contradiction.

The way we decompose f is to express it as a sum of functions of the form ˜ g ( x ) in III . While the actual decomposition requires a bit more adjustment, it follows from the idea to decompose f in the form

where the index set I is the set of vertices of T with height slightly greater than h A k (to ensure correlation decay). Each f w is a function with variable x S for S ⊆ { u ∈ L : u ≤ w } satisfies the description of ˜ g ( x ) in III .

Observe that this decomposition satisfies the description in II :

The size of I is bounded by 2 ℓ +2 ℓ -1 + · · · 1 ≤ 2 ℓ +1 .

The second moment decay property of f w follows from III .

̸

̸

Finally, consider u, v ∈ I . If v < u , say v ≤ u 1 , then f u and f v satisies the covariance bound condition stated in I with f α = f u , f β = f 2 , and w ′ = u 2 . The case when u < v is similar. When u and v are not comparable, then following the Markov Property, E [ f u ( X ) f v ( X )] = EE [ f u ( X ) | X <w ] E [ f v ( X ) | X <w ] with w being the nearest common ancestor of u and v and X <w = ( X w ′ ) w ′ <w , which makes it easier to show the covariance bound. (See Figure 4 for an illustration.)

̸

̸

w

u

u

n

2

2

With this desirable decomposition, one could try to apply some argument similar to the degree-1 case to show the second moment decay (6) for mean 0 A k +1 -polynomials with a slightly bigger penalty constant h A k +1 than h A k .

V. Overview on the induction Given the decomposition of f as described in IV , together with the second moment assumption (6) on A k -polynomials described in III , the proof of the main theorem proceeds by induction. The goal is to show that the penalty constant h A k associated with A k , which appeared (6), satisfies the following recursive inequality:

for some constant C depending on M and dλ 2 . If true, by taking k to be proportional to ℓ/C , then the theorem follows. The proof of the theorem requires a careful analysis of the covariance and variance decay to demonstrate that it resembles the behavior observed in the degree-1 case, in order to capture the Kesten-Stigum bound.

Comparison to other work in low-degree polynomials While some high-level ideas align with previous work in low degree polynomials mentioned fore, our approach focuses on establishing lowdegree lower bounds in a setting where direct comparisons are challenging due to structural differences. Specifically, our work addresses the broadcasting process on trees, where the underlying structures are highly correlated and do not naturally lend themselves to a product measure representation, presenting unique technical challenges not encountered in more independent setups.

Acknowledgments

Han Huang was supported by Elchanan Mossel's Vannevar Bush Faculty Fellowship ONR-N0001420-1-2826 and by Elchanan Mossel's Simons Investigator award (622132). Elchanan Mossel was partially supported by Bush Faculty Fellowship ONR-N00014-20-1-2826, Simons Investigator award (622132), ARO MURI W911NF1910217 and NSF award CCF 1918421.

References

Emmanuel Abbe. Community detection and stochastic block models: recent developments. The Journal of Machine Learning Research , 18(1):6446-6531, 2017.

Emmanuel Abbe and Colin Sandon. Proof of the achievability conjectures for the general stochastic block model. Communications on Pure and Applied Mathematics , 71(7):1334-1406, 2018.

Afonso Bandeira, Amelia Perry, and Alexander S. Wein. Notes on computational-to-statistical gaps: predictions using statistical physics. Port. Math. , 75(2), 2018.

Afonso S Bandeira, Dmitriy Kunisky, and Alexander S Wein. Computational hardness of certifying bounds on constrained pca problems. In ITCS , 2020.

Jess Banks, Cristopher Moore, Joe Neeman, and Praneeth Netrapalli. Information-theoretic thresholds for community detection in sparse networks. In Conference on Learning Theory , pages 383-416. PMLR, 2016.

P. M. Bleher, J. Ruiz, and V. A. Zagrebnov. On the purity of the limiting Gibbs state for the Ising model on the Bethe lattice. J. Statist. Phys. , 79(1-2):473-482, 1995.

Charles Bordenave, Marc Lelarge, and Laurent Massoulié. Non-backtracking spectrum of random graphs: community detection and non-regular ramanujan graphs. In Foundations of Computer Science (FOCS), 2015 IEEE 56th Annual Symposium on , pages 1347-1357. IEEE, 2015.

C. Borgs, J. Chayes, E. Mossel, and S. Roch. The kesten-stigum reconstruction bound is tight for roughly symmetric binary channels. In Proceedings of IEEE FOCS 2006 , pages 518-530, 2006.

Guy Bresler and Brice Huang. The algorithmic phase transition of random k -sat for low degree polynomials. In FOCS , 2021.

Wei-Kuo Chen, David Gamarnik, Dmitry Panchenko, and Mustazee Rahman. Suboptimality of local algorithms for a class of max-cut problems. The Annals of Probability , 47(3):1587-1618, 2019.

A. Decelle, F. Krzakala, C. Moore, and L. Zdeborová. Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Physics Review E , 84:066106, Dec 2011.

Aurelien Decelle, Florent Krzakala, Cristopher Moore, and Lenka Zdeborová. Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Physical Review E , 84(6):066106, 2011.

Yash Deshpande and Andrea Montanari. Finding hidden cliques of size √ N/e in nearly linear time. Foundations of Computational Mathematics , 15(4):1069-1128, 2015.

W. S. Evans, C. Kenyon, Yuval Y. Peres, and L. J. Schulman. Broadcasting on trees and the Ising model. Ann. Appl. Probab. , 10(2):410-433, 2000.

David Gamarnik, Aukosh Jagannath, and Alexander S Wein. Low-degree hardness of random optimization problems. In 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS) , pages 131-140. IEEE, 2020.

David Gamarnik and Madhu Sudan. Limits of local algorithms over sparse random graphs. In Proceedings of the 5th conference on Innovations in theoretical computer science , pages 369-376, 2014.

Justin Holmgren and Alexander S Wein. Counterexamples to the low-degree conjecture. In ITCS , 2020.

Samuel Hopkins. Statistical inference and the sum of squares method . PhD thesis, Cornell University, 2018.

Samuel Hopkins and David Steurer. Efficient bayesian estimation from few samples: community detection and related problems. In 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS) , pages 379-390. IEEE, 2017.

D. Ioffe. Extremality of the disordered state for the Ising model on general trees. In Trees (Versailles, 1995) , volume 40 of Progr. Probab. , pages 3-14. Birkhäuser, Basel, 1996.

D. Ioffe. On the extremality of the disordered state for the Ising model on the Bethe lattice. Lett. Math. Phys. , 37(2):137-143, 1996.

Harry Kesten and Bernt P Stigum. Additional limit theorems for indecomposable multidimensional galton-watson processes. The Annals of Mathematical Statistics , 37(6):1463-1481, 1966.

Frederic Koehler and Elchanan Mossel. Reconstruction on trees and low-degree polynomials. Advances in Neural Information Processing Systems , 35:18942-18954, 2022.

D. Kunisky, A.S. Wein, and A.S. Bandeira. Notes on computational hardness of hypothesis testing: Predictions using the low-degree likelihood ratio. Mathematical Analysis, its Applications and Computation. ISAAC 2019. Springer Proceedings in Mathematics and Statistics , 385, 2022.

Dmitriy Kunisky, Alexander S Wein, and Afonso S Bandeira. Notes on computational hardness of hypothesis testing: Predictions using the low-degree likelihood ratio. arXiv preprint arXiv:1907.11636 , 2019.

Cheng Mao and Alexander S Wein. Optimal spectral recovery of a planted vector in a subspace. arXiv preprint arXiv:2105.15081 , 2021.

M. Mézard and A. Montanari. Reconstruction on trees and the spin glass transition. Journal of Statistical Physics , 124:1317-1350, 2006.

Sidhanth Mohanty, Siqi Liu, and Prasad Raghavendra. On statistical inference when fixed points of belief propagation are unstable. In IEEE 62st Annual Symposium on Foundations of Computer Science (FOCS) , 2021.

Andrea Montanari and Alexander S. Wein. Equivalence of approximate message passing and low-degree polynomials in rank-one matrix estimation. Probability Theory and Related Fields , 2024.

E. Mossel. Reconstruction on trees: beating the second eigenvalue. Ann. Appl. Probab. , 11(1):285-300, 2001.

E. Mossel. Phase transitions in phylogeny. Trans. Amer. Math. Soc. , 356(6):2379-2404 (electronic), 2004.

E. Mossel. Survey: Information flow on trees. In J. Nestril and P. Winkler, editors, Graphs, Morphisms and Statistical Physics. DIMACS series in discrete mathematics and theoretical computer science , pages 155-170. 2004.

E. Mossel, J. Neeman, and A. Sly. Reconstruction and estimation in the planted partition model. Probability Theory and Related Fields , (3-4):431-461, 2015. The Arxiv version of this paper is titled Stochastic Block Models and Reconstruction.

Elchanan Mossel. Combinatorial statistics and the sciences. In International Congress of Mathematicians: 2022 July 6-14 , volume 6, chapter 5553, pages 1-20. 2023.

Elchanan Mossel, Joe Neeman, and Allan Sly. A proof of the block model threshold conjecture. Combinatorica , 38(3):665-708, 2018.

Elchanan Mossel and Yuval Peres. Information flow on trees. The Annals of Applied Probability , 13(3):817-844, 2003.

Elchanan Mossel, Allan Sly, and Youngtak Sohn. Exact phase transitions for stochastic block models and reconstruction on trees. In STOC 2023: Proceedings of the 55th Annual ACM Symposium on Theory of Computing , pages 96-102, 2023.

Federico Ricci-Tersenghi, Guilhem Semerjian, and Lenka Zdeborová. Typology of phase transitions in bayesian inference problems. Physical Review E , 99(4):042109, 2019.

A. Sly. Reconstruction of random colourings. Comm. Math. Phys. , 288, 2009.

Alexander S Wein. Optimal low-degree hardness of maximum independent set. arXiv preprint arXiv:2010.06563 , 2020.

Lenka Zdeborova and Florent Krzakala. Statistical physics of inference: Thresholds and algorithms. Advances in Physics , 65(5), 2016.

Appendix

Overview

In Section A, we give additional notations and basic tools.

In Section B, we formulate the main theorem we want to prove as an induction statement.

In Section C, we discuss the case for degree 1 polynomial, which prove the base case of the induction in the theorem, and the results for degree 1 polynomial will be used in the inductive step.

In Section D, we give a procedure to decompose B -polynomial f for a given collection B .

In Section E and F, we derive the proof of Theorem B.6, the inductive step for proving Theorem B.1.

In Section G and H, we derive the main result in the general case.

In Section I, we provide a proof of Proposition C.3, which is one technical obstacle for getting our main result from Theorem B.1 to the general setting (Theorem 1.6). It is postponed to this section due to the proof is essentially a result about Markov Chain.

In Section J, we provided some standard result for decay of Markov Chain.

A Additional Notations and Tools

Let us begin with the proof of Remark 1.15 which shows A k contains all non-empty subsets of L of size ≤ k :

Proof of Remark 1.15. The proof will be carried out by induction on k . The base case with k = 1 follows from the definition A 1 := { { v } : v ∈ L } . Suppose the claim holds up to some positive integer k . Let ∅ ̸ = S ⊆ L of size | S | ≤ k +1 . If | S | ≤ k , then S ∈ A k ⊆ B ( A k ) = A k +1 . In the case | S | = k +1 ≥ 2 , we first observe that ρ ( S ) is not a leave. Consider the branch decomposition of S (See Definition 1.9):

Because | I ( S ) | > 1 , for each i ∈ I ( S ) we have | S i | < | S | = k + 1 . Therefore, S i ∈ A k for i ∈ I ( S ) , which in turn implies S ∈ B ( A k ) = A k +1 . Therefore, the claim follows.

Now, to show the second statement. For every node w , let S w = { v ∈ L : v ≤ w } . Observe that if w is k -1 layers above L , then S w are the ( k -1) th descendants of w , which has size | S w | = d k -1 .

We claim that for w which are k -1 layers above L , then S w ∈ A k . Let us prove the claim by induction. First, it is clear that for w ∈ L , S w = { w } ∈ A 1 . Suppose the statement holds up to k . Take any w which is k layer above L . Then, the branch decomposition of S w = ⊔ i ∈ [ d ] S w i . With each w i is k -1 layer above L , we have S w i ∈ A k . Hence, S w ∈ A k +1 due to A k +1 = B ( A k ) .

A.1 Additional notations

For any positive integer n , let [ n ] denote the set of positive integers from 1 to n , inclusive: [ n ] = 1 , 2 , . . . , n . For integers a and b where a < b , let [ a, b ] = { a, a +1 , . . . , b } .

Additional Notation for Trees

For u ∈ T , we define

to be the set of k th descendants of u . For brevity, let L k := L k ( ρ ) . Further, for h ∈ [0 , h( u )] , let

namely the set of descendants of u which has height h . Observe that

Last, let where

(See Figure 5 for an illustration.)

Definition A.3. [Conditional Expectation]

For each antichain U ⊆ T and f : [ q ] T → R let

To rephrase it, the function ( E U f )( x ) represents the expected value of f ( X ) condition on X v = x v for all vertices v that are not descendents of any u ∈ U . In the case when U = { u } , we will abuse the notation and denote E u as E { u } for u ∈ T . Futher, for any h ∈ [0 , ℓ ] , we set

Remark A.4 (Conditional Expectation and the Markov Property) . Suppose f is a function of x L and U an antichain of T . Let

is a function of x U and x L ′ . To see that, by the Markov Property, x <U and x ̸≤ U are independent conditioned on x U . Thus,

which implies E U f is a function of x U and x L ′ . (See Figure 6 for an illustration.)

T u be the induced subgraph of T with vertex set { v ∈ T : v ≤ u } and L u = T u ∩ L . It is worth noting that T u can be seen as a rooted tree with root u and h( u ) layers, and the h( u ) th layer is L u .

Additional Notation for collection A ⊆ 2 L \ {∅}

Definition A.1. For a given collection of of subsets A ⊆ 2 L \{∅} , we define the following subcollections: For each u ∈ T , let A u := { S ∈ A : ρ ( S ) = u } , A ≤ u := { S ∈ A : ρ ( S ) ≤ u } , and A <u := { S ∈ A : ρ ( S ) < u } .

Notation for Conditional Expectation

Definition A.2. Recall that an antichain U ⊆ T is a collection of vertices such that no two vertices in U are comparable under the ≤ relation. For x ∈ [ q ] T , we can decompose x in the form

and

And from the above two inequalities, adjusting the constant C if necessary, we also have

In this paper, we will fix a basis for the space of functions from [ q ] to R for a Markov chain M .

Definition A.6. For a given q × q ergodic and irreducible transition matrix M , we fix a basis { ϕ i } i ∈ [0 ,q -1] for the space of functions from [ q ] to R such that ϕ 0 is the constant function 1 and ϕ i for i ∈ [ q -1] are functions such that

Definition A.7. Suppose a given ℓ layer rooted tree T and q × q transition matrix described in Lemma A.5 have been given. For σ ∈ [0 , q -1] L , let

̸

the set of vertices in L in which σ is non-zero. Let | σ | = | S ( σ ) | . When | σ | ≥ 1 , we define

And when | σ | ≥ 2 , we define

Further, let

Remark A.8. We remark that ϕ σ ( x ) is a function with variables x S ( σ ) .

The fact that ϕ 0 , ϕ 1 , . . . , ϕ q -1 forms a basis implies that:

and

A.2 Basis Functions on [ q ] L and some decay properties

The following lemma is a well-known statement from Markov-Chain. Let us formulate it using the Broadcasting Process. (For completeness, we include a proof of this lemma in Section J.)

Lemma A.5. Suppose M is irreducible and aperiodic, then there exists C = C ( M ) > 1 so that the following hold: For any u ∈ T and k ∈ N so that p k ( u ) = p ◦ p ◦ . . . p ︸ ︷︷ ︸ k times ( u ) , the k th ancester of u , exists. For every function a with variable x u ,

Fact A.9. Every function f : [ q ] U → R can be expressed uniquely in the form

̸

Remark A.10. With the above representation, the Efron-Stein degree of function f equals to the largest magnitude of | σ | among those σ such that c σ = 0 .

Definition A.11. Given a tree T and a q × q ergodic transition matrix M , let { ϕ i } i ∈ [ q -1] be the functions described in Lemma A.5. For a collection of subsets A ⊆ 2 L \{∅} , let

For any σ ∈ F ( A ) with | σ | > 1 , let

where, for each i ∈ I ( σ ) , P i σ is the restriction of σ to S ( σ ) i :

B The overall inductive argument

Here we present the version of the theorem with additional assumption on the transition matrix M that

Theorem B.1. Given the rooted tree T and the transition matrix M described in Theorem 1.6, and under the additional assumption that c M = min i,j ∈ [ q ] M ij > 0 , there exists c > 0 dependent on M and dλ 2 (and implicitly on c M as well) so that the following holds: For any function f of the leaves with fractal capacity ≤ c ℓ log( dR ) ,

We will first derive the version mentioned above, as it substantially reduces the technical complexity without compromising the structural integrity of the proof in the general setting where c M might be 0 .

The proof of Theorem 1.6 will be carried out by induction on A k -polynomials. Let us introduce the necessary notations to outline this induction process.

Definition B.2. Let ε > 0 be the constant such that

The constant ε is introduced to improve the readability of the paper. Intuitively, we aim to define dλ 2 = exp( -ε ) , but we relax this definition slightly so that inequalities like the following hold when ℓ is sufficiently large:

Assumption B.3. We say that A satisfies assumption B.3 with parameters (h ∗ , c ∗ ) where h ∗ > 0 and 0 < c ∗ < 1 , if

is closed under decomposition , and morever,

For any v ∈ T with h( v ) ≥ h ∗ and a A ≤ v -polynomial f ,

The inequality (15) bears a resemblance to the inequality we aim to prove in Theorem B.1. The second inequality, (16), will later be seen as a crucial step proving the inductive phase of our proof. Indeed, in the case where c M > 0 , the condition (16) can be easily satisfied by appropriately choosing c ∗ .

Lemma B.4. For any given A 1 ⊆ A ⊆ 2 L \{∅} which is closed under decomposition. If it satisfies (15) with a given parameter h ∗ and c M > 0 , then A satisfies Assumption B.3 with parameter h ∗ and

In other words, we can choose c ∗ with no dependence on either h ∗ or A .

Proof. Consider an arbitrary function f with variables ( x u : u ≤ v ) for some v ∈ T \{ ρ } .

Let g ( x ) := ( E v f 2 )( x ) . By the Markov Property, ( E v f 2 )( x ) is a function of x v , which in turn implies g ( x ) = g ( x v ) . Now, for any θ ∈ [ q ] , fix an index j 0 ∈ [ q ] such that g ( j 0 ) ≥ E g ( X v ) . Relying on g is a non-negative function,

By unraveling the definition of g , we can satisfy the first inequality of (16) as long as c ∗ < c M . The proof for the second inequality follows a similar logic, using the condition c ∗ ≤ 1 min j π ( j ) and the trival inequality max i,j M ij ≤ 1 .

Given this notation, the proof of Theorem B.1 proceeds by induction, with the base case and inductive articulated in the subsequent two statements.

Proposition B.5. Given the rooted tree T and the transition matrix M described in Theorem 1.6, and under the additional assumption that c M = min i,j ∈ [ q ] M ij > 0 . There exists C = C ( M,ε ) ≥ 1 so that the following holds:

Fix ρ ′ ∈ T and 0 ≤ m ≤ h( ρ ′ ) , if f ( x ) is a degree 1 polynomials of variables ( x v : v ∈ D m ( ρ ′ )) , then

Theorem B.6. Given the rooted tree T and the transition matrix M described in Theorem 1.6, and under the additional assumption that c M = min i,j ∈ [ q ] M ij > 0 . Suppose A is a collection of subsets satisfying Assumption B.3 with parameters (h ∗ , c ∗ ) . Then, there exists C = C ( M,d,c ∗ ) ≥ 1 such that B = B ( A ) satisfies Assumption B.3 with parameters ( h ∗ + C (log( R ) + 1) , c ∗ ) .

Let us derive the proof of Theorem B.1 based on the above two statements.

Proof of Theorem B.1. We apply Proposition B.5 and Lemma B.4 to get A 1 satisfies Assumption B.3 with parameter h ∗ = C B. 5 (log( R ) + 1) , where C B. 5 = C ( M,d ) is the constant introduced in the Proposition and

Then, by applying Theorem B.6 inductively on the chain A k , we can conclude that A k satisfies Assumption B.3 with parameter h ∗ = C (log( R ) + 1) k and the same c ∗ described above, provided that C = C ( M,d,c ∗ ) is the maximum of the constants C described in Proposition B.5 and Theorem B.6. In other words, for any A k -polynomial f ,

The theorem follows by choosing k = 1 2 C (log( R )+1) ℓ .

C Variance Decomposition and Variance Estimate for degree 1 polynomials

To describe the goal of this section, let us begin with the variance decomposition of degree 1 polynomials in a slight generalized form. Essentially, the following statement is a direct consequence of the conditional variance formula. We will state the main results first and provide the proof later in this section.

Lemma C.1. Fix ρ ′ ∈ T and 0 ≤ k ≤ h( ρ ′ ) , consider a function g : [ q ] T → R of the form

For w ∈ T ρ ′ \{ ρ ′ } with h( w ) ≥ k +1 , let

Then,

Our goal is to show that when dλ 2 < 1 , Var[ g ( X )] is of the same order as ∑ v ∈ D k ( ρ ′ ) Var[ g v ( X )] , based on the above lemma.

Lemma C.2. Suppose the transition matrix M satisfies dλ 2 < 1 and the tree T has growth factor R . Then, there exists a constant C = C ( M,ε ) ≥ 1 so that the following holds. Let ρ ′ ∈ T , l ′ := h( ρ ′ ) , and k ∈ [0 , l ′ ] . Consider a function of the form g ( x ) = ∑ v ∈ D k ( ρ ′ ) g v ( x v ) . Then,

The opposite bound does not depend on dλ 2 ≤ 1 . However, the proof in the general case where c M = 0 is not straight-forward. We state it in full generality but will defer the general proof and prove it here in the simpler case where c M > 0 .

Proposition C.3. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds. Let ρ ′ ∈ T , and k ∈ [0 , h( ρ ′ )] . For any degree-1 function g with variables ( x v : v ∈ D k ( ρ ′ )) . There exists functions g v ( x ) = g v ( x v ) for v ∈ D k ( ρ ′ ) so that the following holds:

1. g ( X ) = ∑ v ∈ D k ( ρ ′ ) g v ( X u ) almost surely. (They may not agree as functions from [ q ] T to R .)

For any u ∈ T ρ ′ with h( u ) ≥ k ,

In particular, taking u = ρ ′ we have

We postpone the proof of Proposition in full generality in Appendix I, due to the technical complexity of the proof and the fact that the proof is about properties of a Markov Chain. Instead, a statement of the proposition and its proof in the case where c M > 0 is provided in this section.

Now, the purpose of this section is twofold.

First, it is the derivation of the variance related estimates: Lemma C.1, Lemma C.2, and Proposition C.3 with the additional assumption that c M > 0 . Additionally, we summarise the estimates into a single statements, as stated in Lemma C.7.

Second, it is the derivation of the base case of the induction, Proposition B.5.

C.1 Variance Decomposition and Estimates

Before we proceed to the proof of Lemma C.1, let us remark on the following consequence of the lemma.

Remark C.4. For any g described in Lemma C.1, if we define h ( x ) := ( E k g )( x ) = ∑ v ∈ D k ( ρ ′ ) ( E v g v )( x v ) and h v ( x ) := ( E v g v )( x v ) , then by applying the lemma to both g and h , we conclude that

Proof of Lemma C.1 . First, for v ∈ T ρ ′ \{ ρ ′ } with h( v ) ≥ k ,

Let us start decomposing the variance of g .

By the Markov Property and rearrangement of the terms, for each w ∈ T ρ ′ with h( w ) > k ,

Hence,

Notice the following holds:

Next, let us show the proof of Lemma C.2.

Proof of Lemma C.2. Let C 0 = C 0 ( M,d ) denote the constant introduced in the statement of the Lemma. Its precise value will be determined along the proof. Without lose of generality, we may assume both E g ( X ) = 0 and E g v ( X v ) = 0 for v ∈ D k ( ρ ′ ) , and the variance of each function is simply its the second moment. For brevity, let τ u := ( E ( g u ( X )) 2 ) 1 / 2 for u ∈ D k ( ρ ′ ) and ℓ ′ := h( ρ ′ ) .

By (8) from Lemma A.5, for s ∈ [ l ′ -k ] ,

where C A. 5 ≥ 1 is the constant introduced in the Lemma. In particular, if ρ ( u, u ′ ) = p s ( u ) for u, u ′ ∈ D k ( ρ ′ ) , then

Then,

where the sum ∑ u,u ′ is taken over all ordered pairs ( u, u ′ ) with u, u ′ ∈ D k ( v ) with ρ ( u, u ′ ) = v . By relaxing the constraint of the summation we have

Next,

Hence, C 0 depends on ε , q , and C A. 5 . It is a constant which is determined by M and ε .

Let us formulate Proposition C.3 under the additional assumption that c M = min i,j ∈ [ q ] M ij > 0 . Indeed, in this case, the bound does not depend on R .

Proposition C.5. Suppose the transition matrix M satisfies c M > 0 . There exists a constant C = C ( M,ε ) ≥ 1 so that the following holds: Let ρ ′ ∈ T , l ′ := h( ρ ′ ) , and k ∈ [0 , l ′ ] . For any function g = [ q ] T → R of the form

The following holds: For any u ∈ T ρ ′ with h( u ) ≥ k ,

In particular, taking u = ρ ′ we have

The proof of the Proposition relies on the following immediate consequence of c M > 0 :

Lemma C.6. Suppose M is a q × q ergodic transition matrix with c M = min i,j ∈ [ q ] M ij > 0 . There exists C = C ( M ) ≥ 1 so that the following holds. For any u ∈ T \{ ρ } and a function h ( x ) = h ( x u ) ,

Proof. Let θ 1 = argmin θ ∈ [ q ] h ( θ ) and θ 2 = argmax θ ∈ [ q ] h ( θ ) . (In the case of a tie, we may choose any of the minimizers or maximizers.) First, we have

Next, for any β ∈ [ q ] , we have

Let i ∈ { 1 , 2 } be the index such that | E [ h ( X u ) | X p ( u ) = β ] -h ( θ i ) | ≥ 1 2 | h ( θ 2 ) -h ( θ 1 ) | , and we will use this together with c M > 0 to give a lower bound on the conditional variance:

Since it holds for every β ∈ [ q ] , we conclude that

Proof of Proposition C.5. We adapt the notation from Lemma C.1. For w ≤ ρ ′ with h( w ) > k , let

Now, we apply Lemma C.1 and Lemma C.6 to get

where we used the fact that all the terms are non-negative, the first inequality is obtained by looking at the second terms for the summands with h ( w ) = k and C C. 6 is the M -dependent constant introduced in Lemma C.6.

Lemma C.7. Suppose dλ 2 < 1 and the growth factor is at most R . There exists a constant C = C ( M,d ) ≥ 1 so that the following holds. Fix ρ ′ ∈ T and 0 ≤ m ≤ h( ρ ′ ) , if f m ( x ) is a function in the form

Then, there exists ¯ f v ( x ≤ v ) for v ∈ D m ( ρ ′ ) such that their sum ¯ f m ( x ) = ∑ v ∈ D m ( ρ ′ ) ¯ f v ( x ≤ v ) satisfies and for u ≤ ρ ′ with h( u ) ≥ m ,

The statement of the lemma using ¯ f m and ¯ f b covers also the case c M = 0 . We will prove the Lemma by using either Proposition C.3 or Proposition C.5 with the assumption c M > 0 . In the later case, it suffice to simply take f v ( x ≤ v ) = ¯ f v ( x ≤ v ) .

Remark C.8. Note that the lemma implies the following: For any u ≤ ρ ′ with h( u ) ≥ m , let

Then, for any given m ≤ k < k ′ ≤ h( ρ ′ ) and u ∈ D k ′ ( ρ ′ ) , we have

where the first inequality follows from the second inequality of (24) and the second inequality follows from the first inequality of (24).

Proof of Lemma C.7. Let

with and

Note that h is a degree one function of the variables ( x v : v ∈ D m ( ρ ′ )) . Thus, we could apply Proposition C.3 to show the existence of 1-variable functions h v ( x v ) for v ∈ D m ( ρ ′ ) such that

and for any u ∈ T ( ρ ′ ) with h( u ) ≥ m ,

where C C. 3 ≥ 1 is the constant introduced in Proposition C.3.

Since E h ( X ) = E ( E m f m )( X ) = 0 , we may also assume that

for v ∈ D m ( ρ ′ ) , as a constant shift of the functions will not affect (26) and (27). Now, consider the following functions: For v ∈ D m ( ρ ′ ) , let

First, since ¯ f v ( x ≤ v ) is defined as the sum of three terms with mean 0 , we have E ¯ f v ( X ≤ v ) = 0 . Second,

To estimate the lower bound, we rely on own choice of h v . By (27) we have

As for the upper bound, we can apply Lemma C.2 and repeat the same derivation as above to get

Therefore, by taking the constant C stated in the lemma to be the maximum of C C. 3 and C C. 2 , the proof follows.

C.2 Proof of the base Case of Proposition B.5

We now prove the base case of Proposition B.5:

Lemma C.9. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds. For any degree 1 function f with variables ( x u : u ∈ D k ( ρ ′ )) for some ρ ′ ∈ T with k ≤ h( ρ ′ ) ,

where h ′ = h( ρ ′ ) -k .

Proof. Let f u for u ∈ D k ( ρ ′ ) be the functions from Proposition C.3 so that

We can assume E f ( X ) = 0 and E f u ( X ) = 0 for every u ∈ D k ( ρ ′ ) without affecting (30). From Proposition C.3, we have

where C C. 3 denotes the constant C introduced in the Proposition.

We could apply Lemma A.5 to get

where C A. 5 denotes the constant C stated in the Lemma.

Proof of the Base Case Proposition B.5. Given ρ ′ ∈ T and 0 ≤ m ≤ h( ρ ′ ) described in the Proposition. Let h ′ = h( ρ ′ ) -m . By Lemma C.9, any function f ( x ) = ∑ v ∈ D m ( ρ ′ ) f v ( x v ) satisfies

where C C. 9 denotes the M -dependent constant introduced in the Lemma. With

C C. 9 R 4 (h ′ ) q ( dλ 2 ) h ′ ≤ C C. 9 R 4 (h ′ ) q exp( -1 . 1 ε h ′ ) ≤ exp ( -ε (h ′ -C 1 (log( R ) + 1)) ) , for some C 1 ≥ 1 which depends on M,d .

D Decomposition of Polynomials

In this section we study the representation of functions in terms of ϕ σ and ψ σ . Roughly speaking ψ σ are 'more orthogonal" than the ϕ σ . More formally we will show that under appropriate conditioning expections of ψ σ factorize. Thus some of the effort in the proof and particularly in this section is devoted to relating the ϕ and ψ representations and bounding moments of such representations.

Lemma D.1. Assuming dλ 2 < 1 and growth factor of R , there exists C = C ( M,d ) ≥ 1 so that the following holds. Let A 1 ⊆ B ⊆ 2 L \{∅} be a collection of subsets which is closed under decomposition. Fix a positive integer k 1 and ρ ′ ∈ T with l ′ := h( ρ ′ ) > k 1 . For every function f of the form

̸

with here exists a decomposition of f

where, for each u ≤ ρ ′ with h( u ) ≥ k 1 , we have a function f u ( x ) = f u ( x ≤ u ) and

For u ∈ T ρ ′ with h( u ) > k 1 , f u ( x ) is a linear combination of ψ σ ( x ) with σ ∈ F ( B u ) and ˜ f u ( x ) = f u ( x ) -E f u ( X ) .

For w ≤ ρ ′ with h( w ) ≥ k 1 , we have

We may group the f u according to h( u ) and define for k 1 ≤ k ≤ h( ρ ′ ) ,

To prove the main lemma, let us begin by comparing ϕ σ ( x ) and ψ σ ( x ) (See Definition A.11).

Lemma D.2. For σ ∈ F ( B u ) , ϕ σ ( x ) can be expressed in the form

where:

In other words,

a ⊂ ,σ ( x ) is a linear combination of ϕ σ ′ ( x ) for σ ′ ∈ F ( B u ) such that I ( σ ′ ) is a proper subset of I ( σ ) .

a <,σ ( x ) is a linear combination of ϕ σ ′ ( x ) for σ ′ ∈ F ( B < u ) (recall that B <u = { S ∈ B : ρ ( S ) < u } ), and

a c,σ is a constant.

Proof. Fix σ ∈ F ( B ) and let u = ρ ( S ) and S = S ( σ ) . Recall that P i σ ∈ [0 , q -1] T is the projection of σ to S i . We can also decompose the function ϕ σ according to { P i σ } i ∈ I ( σ ) :

Before we proceed, let us note that by Lemma 1.12 and the definition of B , we have P i σ ∈ F ( B ≤ u i ) . Now, let us expand the function ψ σ according to its definition:

where the summation is taken over all possible partition I 1 ⊔ I 2 = I ( σ ) . Next, we can group the summands into four types based on | I 1 | and | I 2 | :

Type 1 | I 1 | = | I ( σ ) | . The summand is simply ϕ σ ( x ) .

Type 2 2 ≤ | I 1 | ≤ | I ( σ ) | -1 .

Each summand is a constant multiple of ϕ σ ′ ( x ) where σ ′ is the projection of σ to the indices ⊔ i ∈ I 1 S i . Clearly, S ( σ ′ ) = ⊔ i ∈ I 1 S i . With | I 1 | ≥ 2 , we have ρ ( σ ′ ) = u . Further, each S i ∈ A ≤ u i for i ∈ I ( σ ) , it follows that S ( σ ′ ) ∈ B u , which in turn implies σ ′ ∈ F ( B u ) .

We denote the sum of summands of this type by a ⊂ ,σ ( x ) .

Type 3 | I 1 | = 1 . Each summand is a constant multiple of ϕ P i σ ( x ) , where i is the element in I 1 . Notice that P i σ ∈ F ( A <u ) ⊂ F ( B <u ) where the inclusion follows from Lemma 1.12. We denote the sum of summands of this type as a <,σ ( x ) .

Type 4 | I 1 | = 0 There is only one summand, which is a constant. We denote this constant by a c,P i σ .

With this decomposition, (32) follows.

Given the expressions for ψ σ ( x ) in terms ϕ σ ( x ) and vice-versa, for any given u ∈ T \ L , we can convert a linear combination of ϕ σ ( x ) with σ ∈ F ( B u ) to that of ψ σ ( x ) with σ ∈ F ( B u ) .

Lemma D.3. For u ∈ T \ L , consider any function of the form

˜ f u ( x ) = f u ( x ) -E f u ( X ) and f u ( x ) is a linear combination of ψ σ ( x ) for σ ∈ F ( B u ) ,

p <,u ( x ) is a linear combination of ϕ σ ( x ) with σ ∈ F ( B <u ) , and

c u is a constant.

Proof. The decomposition is constructed through recursion on the following expression:

̸

Suppose r ( p u ) = 2 . Then the statement of simply follows from Lemma D.2.

Suppose the statement of the lemma holds whenever r ( p u ) ≤ r for 2 ≤ r < Rd . Consider any function p u with r ( p u ) = r +1 :

According to the decomposition of ϕ σ ( x ) in Lemma D.2, let

where ∗ can be ⊂ , < , or c . Then,

Observe that p u, ≤ r ( x )+ p ⊂ ,u,r +1 ( x ) is a linear combination of ϕ σ ( x ) with σ ∈ F ( B u ) and | I ( σ ) | ≤ r . Thus, by the inductive assumption, the summation can be expressed in the form

Then there exists a decomposition where:

Finally, let and we have

Proof of Lemma D.1. We will construct f u ( x ) for u starting from top layer ( u = ρ ′ ) to bottom layer. For k ∈ [ k 1 , l ′ -1] , when f u ( x ) is constructed for u ∈ T ρ ′ with h( u ) > k , we define

where ˜ f u ( x ) = f u ( x ) -E f u ( X ) . Without lose of generality, let f ≤ l ′ ( x ) = f ( x ) . Fix k ∈ [ k 1 + 1 , l ′ ] . For the induction step, suppose { f u ( x ) } u ∈ T ρ ′ : h( u ) >k { f ≤ s ( x ) } s ∈ [ k,l ′ ] have been constructed such that f ≤ k ( x ) can be expressed in the form

Clearly, this holds when k = l ′ .

For each u with h( u ) = k , let p u ( x ) = ∑ σ ∈F ( B u ) c ′ σ ϕ σ ( x ) , and define ˜ f u ( x ) , p <,u ( x ) , and c u according to Lemma D.3. Then,

Recall from Lemma D.3 that p <,u ( x ) is a linear combination of ϕ σ ( x ) with σ ∈ F ( B <u ) , the function f ≤ k -1 ( x ) satisfies (36) as well (with k been replaced by k -1 ).

Once the induction terminated at layer k 1 , we obtain

Now, observe that for k 1 < k ≤ h( ρ ′ ) , we have f k ( x ) is defined as the sum of ˜ f u for u ∈ D k ( ρ ′ ) , which are functions of mean 0 . Together with the assumption that E f ( X ) = 0 , we have

Notice that f k 1 satisfies the assumption of the function stated in Lemma C.7 with m = k 1 . By replacing f k 1 by ¯ f k 1 and f u by ¯ f u for each u ∈ D k 1 ( ρ ′ ) , the second statement follows while the third statement of the Lemma remains true. Hence, the proof is completed.

E Induction Step 1: Decay of f u

The goal this section and next section is to prove Theorem B.6. Let us restate the theorem here.

Theorem. Given the rooted tree T and the transition matrix M described in Theorem 1.6, and under the additional assumption that c M = min i,j ∈ [ q ] M ij > 0 . Suppose A is a collection of subsets satisfying Assumption B.3 with parameters h ∗ and c ∗ . Then, there exists C = C ( M,d,c ∗ ) ≥ 1 such that B = B ( A ) satisfies Assumption B.3 with parameters h ∗ + C (log( R ) + 1) and c ∗ .

In this and the following section , we will fix a collection A that meets Assumption B.3 with some parameters l ∗ and c ∗ . Additionally, we abbreviate

Further, we will fix ρ ′ ∈ T and a function f described in the Assumption B.3, and assume

The proof is grounded in the decomposition of f as described in Lemma D.1, which splits f into summation of f k and subsequently into summations of ˜ f u . Accordingly, this section is devoted to derive the variance decay properties of f u stated as Proposition E.1 below. The proposition will be used to derive variance decay properties of f k , and toward the proof of Theorem B.6 in next section.

Proposition E.1. There exists C = C ( M,c ∗ ) ≥ 1 so that the following holds. For any u ∈ T , consider a function f u of the form

̸

Then, for θ ∈ [ q ] , we have the following bounds on ( E u f u )( x ) (recall that that by the Markov Property, ( E u f u )( x ) is a function of x u ):

and

Additionally, for any function a ( x ) having inputs involving only ( x v : v ∈ T u i ) for some i ∈ [ d u ] :

Remark E.2. The statement of the Proposition E.1 is exactly the statement of Theorem B.6 restricted to functions all of whose non-zero c σ have ρ ( σ ) = ρ ′ . Thus in some sense in this section we prove the Theorem for the most complex terms. And in the next section we will control the correlations between different terms.

This is an analogue in our setting to the classical fact in Fourier analysis that high amplitude functions have sharp decay under noise.

Remark E.3. We remark that the proposition holds immediately whenever | d u | ≤ 1 , since B u = ∅ .

Before we proceed further, we need to decompose f u ( x ) .

Definition E.4. For u ∈ T \ L and f u ( x ) = ∑ σ ∈F ( B u ) c σ ψ σ ( x ) , let

for each I ⊆ [ d u ] with | I | ≥ 2 .

Given the above definition, we have

Proposition E.5. There exists C = C ( M,c ∗ ) ≥ 1 so that the following holds. For any u ∈ T \ L and I ⊆ [ d u ] with | I | ≥ 2 . Consider a function of the form

Then, for I ′ ⊆ I , let and we have

Roughly speaking the proposition states that under the decay of correlation in Assumption B.3, for functions all of whose coefficient c σ have S ( σ ) = I for some large set I we get a variance decay of the form exp( -ϵ | I | h( u )) . For later applications the statement is more general allowing to condition on some of the subtrees. This is an analogue in our setting to the classical fact in Fourier analysis that high amplitude functions have sharp decay under noise.

Proof. We fix u ∈ T \ L and I ⊆ [ d u ] . Without lose of generality, we assume I ′ = [ s ] .

Let C 0 = C 0 ( M,c ∗ ) denote the constant described in the statement of the Proposition. The precise value of C 0 will be determined during the proof.

For brevity, we introduce some notations that are only used in this proof.

Decomposition of x ∈ [ q ] T : Consider the representation of x as

where, ∀ k ∈ [ s ] , x k := ( x v : v ≤ u k ) , and x 0 = ( x v : v ∈ U \ { u } ) .

Further, let

For k ∈ [0 , s ] ,

Before we proceed to the proof, observe that applying Jenson's inequality on conditional expectation, we can form a chain of inequalities

If h( u ) ≤ C 0 +h ∗ , then the statement of the Proposition is weaker than the inequality ( E U a ) 2 ( x ) ≤ ( E U a 2 )( x ) stated above. So the lemma follows immediately in that case. From now on we assume

We will improve each inequality in the above chain by leveraging the assumption (15). Given the definition of a ( x ) ,

By the Markov Property, the random variables ( X i | X u = x u ) i ∈ [0 ,s ] are independent. This gives rise to:

Now, fix k ∈ [ s ] and express a ≤ k ( x ) = a ≤ k ( x u , x ≤ k -1 , x k ) . An essence of this proof is that the mapping:

is a linear combination of ˜ ϕ σ ( y k ) with σ ∈ F ( A u k ) and the coefficients are functions of ( x u , x ≤ k -1 ) , which gives us room to apply the inductive assumption, or (15) from Assumption B.3.

To aid our analysis, we introduce Y k , an independent copy of X k . By (15), we have

The reason we introduce Y k is that the L.H.S. and R.H.S. of the above inequality are not related to (any moments of) conditional expectation of a ( X ) . However, it can still be used with some adjustment, relying on (16) from Assumption B.3.

Given the assumption on C 0 being greater than or equal to 1 , we have

Applying (16) to our function y k ↦→ a ≤ k ( x u , x ≤ k -1 , y k ) we get

On the other hand,

Combining the above expression, (44), and (45), we conclude that

Notice that the expression inside the square in L.H.S. is

Similarly,

By imposing the first assumption on C 0 that

it follows from (46) that

By taking Conditional Expectation on both sides,

Finally, we apply this inequality consecutively for k ∈ [ s ] we obtain

Corollary E.6. Fix u ∈ T \ L and a function f u ( x ) following the form described in Definition E.4. If I, J ⊆ [ d u ] are subsets of [ d u ] of size at least 2 , then for every θ ∈ [ q ] ,

where C = C ( M,c ∗ ) is the constant introduced in Proposition E.5, and I ∆ J := ( I \ J ) ∪ ( J \ I ) .

Proof. For the first statement, it follows from Proposition E.5 with a ( x ) = f u,I ( x ) and I = I ′ .

To prove the second statement, we begin by noting that the inputs of f u,I ( x ) and f u,J ( x ) do not include ( x v : v ∈ ⋃ i ∈ J \ I T u i ) and ( x v : v ∈ ⋃ i ∈ I \ J T u i ) , respectively. Thus, we can apply the Markov Property and that fact that if Y, Z, W are ind pendent then:

and this in turn becomes:

to obtain

In terms of absolute value, by Proposition E.5 we have

̸

where the second to last inequality follows from Hölder's inequality.

Corollary E.7. There exists C = C ( M,d,c ∗ ) ≥ 1 so that the following holds. If u ∈ T \ L with h( u ) ≥ h ∗ + C (1 + log( R )) , then for any f u ( x ) in the form as described in Definition E.4,

̸

̸

̸

̸

̸

Proof. Let C 0 = C 0 ( M,d,c ∗ ) denote the constant introduced in the statement of the Lemma. Its value will be determined along the proof.

The statement of the Corollary is trivial when d u < 2 since in that case B u = ∅ , implying f u = 0 . From now on, we assume d u ≥ 2 .

First,

where ∑ { I,J } refers to the sum over all unordered pairs { I, J } with I and J being distinct subsets of [ d u ] of size at least 2 .

We can apply (48) to estimate the absolute value of the difference.

where the constant C E. 5 is the constant C E. 5 introduced in Proposition E.5. By 2 | ab | ≤ a 2 + b 2 for a, b ∈ R ,

Hence,

With |{ J ⊆ [ d u ] : | I ∆ J | = i }| = ( d u i ) ≤ d i u ,

̸

provided that h( u ) -C E. 5 -log( d u ) ε -h ∗ ≥ 16 ε .

Now, we impose the first assumption on C 0 that

then

Hence,

and the proof follows.

Proof of Proposition E.1. Let C 0 = C 0 ( M,d,c ∗ ) denote the constant introduced in the statement of the Proposition. Its precise value will be determined along the proof. From Remark E.3, it is sufficient to consider the case when | d u | ≥ 2 . Further, it suffices to prove in the case when

̸

since otherwise the statements follow from either Cauchy-Schwarz or Jenson's inequality.

Part I: Derivation of (37) and (38) .

First, by (47),

where we applied Cauchy-Schwarz inequality in the last inequality; the constant C E. 5 is the constant C introduced in Proposition E.5.

With the coarse estimate we have

The geometric series above is finite if h( u ) is large enough, and this can be achieved by imposing assumption of C 0 and relying on (53). Now, let us impose the first assumption on C 0 :

Then, by (53) we have

which in term implies the R.H.S. of (55) is

Substituting the above estimate into (54), together with (49) we have

Therefore, we have derived an inequality which is slightly stronger than (37).

To derive (38), let us first show E f u ( X ) is relatively small using (57) and Jesnon's inequality:

Thus, the variance and the second moment of f u ( X ) are the same up to a factor of 2 :

We conclude that

Therefore, we complete the proof of (38).

Part II: Derivation of (39) .

It remains to show (39) and the proof is similar. Fix I ⊂ [ d u ] with | I | ≥ 2 , let I ′ = I \{ i } and we represent x ∈ [ q ] T as ( x 0 , x 1 ) , where

With this notation, we have a ( x ) = a ( x 0 ) . Thus,

where the last inequality follows from Proposition E.5. Hence,

Next, we impose the second assumption on C 0 that

where C E. 7 is the constant introduced in Corollary E.7. Together our assumption h( u ) ≥ h ∗ + C 0 (log( R ) + 1) at the beginning of the proof, we can apply the Corollary and (58) to get

Repeating the same argument as in the proof of (38) and relying on the assumption (56) of C 0 ,

Therefore, combining (60), (61), and (59) we get

F Induction Step 2: Decay of f k and the proof of Theorem B.6

As a continuation of the inductive step, we adapt the notation introduced in the previous section. Building on the properties of an single component f u from Proposition E.1, our objective is to deduce variance and covariance decay of f k , which is stated in Proposition F.1 below. Once it is established, we will be ready to prove Theorem B.6.

F.1 Properties of f k

The main goal of this subsection is to derive the following Proposition.

Proposition F.1. There exists C = C ( M,d,c ∗ ) ≥ 1 so that the following holds. For any ρ ′ ∈ T satisfying

Fix a positive integer k 1 such that

with E f ( X ) = 0 . We decompose f according to Lemma D.1 with the given k 1 . Then, the following holds:

for k 1 ≤ m<k ≤ h( ρ ′ ) ,

Consider a function

Before we prove the Proposition, let us prove the following second moment bounds for the partial sums of ˜ f u .

Lemma F.2. There exists a constant C = C ( M,d,c ∗ ) ≥ 1 so that the following holds. Consider the same description as stated in Proposition F .1 and k 1 ≥ h ∗ + C (log( R ) + 1) . Let ( h, k ) be a pair of integers satisfying k 1 < h ≤ k ≤ l ′ . For u ∈ D k ( ρ ′ ) , let

The following holds: First, for u ∈ D k ( ρ ′ ) ,

Second,

Proof. Let C 0 = C 0 ( M,d,ε ′ ) denote the constant introduced in the statement of the Lemma. Its precise value will be determined along the proof.

Let us fix u ∈ D k ( ρ ′ ) . Consider the following conditional expectation of f h,u ( x ) .

̸

̸

where the last inequality follow from Proposition E.1 and C E. 1 is the constant C introduced in Proposition E.1.

Here we impose the first assumption on C 0 :

Then, due to k 1 ≥ h ∗ + C 0 (log( R ) + 1) , we have

and thus (65) can be simplified to

In other words,

With the lower bound been established, the upper bound can also be derived in the same fashion. Let us first recycle the first three lines of (65):

Notice that we can apply Lemma C.2 for the first summand in the above expression.

where C C. 2 is the constant introduced in Lemma C.2. Again, applying the estimate from Proposition E.1 we have

Here we impose the second assumption on C 0 that

Then, relying on h > k 1 ≥ h ∗ + C 0 (log( R ) + 1) ,

which in turn implies

Together with

the second statement of the lemma follows.

Proof of Proposition F.1. Let C 0 = C 0 ( M,d,c ∗ ) denote the constant introduced in the statement of the Lemma. Its precise value will be determined along the proof. Let us make the first assumption on C 0 that

where C F. 2 is the constant introduced in Lemma F.2. Now, we could apply the statements of the Lemma.

Part 1: Derivation of (62) .

Now it remains to show the second statement. Notice that f h = f h,ρ ′ , we immediately have

Fix k ∈ [ k 1 +1 , l ′ ] . Applying Lemma F.2 with the parameters h and k in the Lemma setting to be k ,

The next step is to compare the sum of E ˜ f 2 u ( X ) with E [ ( E ρ ′ f k ) 2 ( X ρ ′ ) ] . By Jenson's inequality,

For each summand, we can apply (8) from Lemma A.5 to get the following estimate.

where C A. 5 is the constant introduced in the Lemma. Together with | D k ( ρ ′ ) | ≤ Rd l ′ -k from the assumption on T and dλ 2 ≤ exp( -1 . 1 ε ) from the definiton of ε ,

where we set

By Proposition E.1 we have

where C E. 1 is the constant C introduced in the Proposition. Substituting this inequality into (69), together with (68) from first step,

Now, we impose the second assumption on C 0 that

we finished the proof of (62).

Part 2: Derivation of (63) .

Let us consider

In other words, we may view h k 1 ( x ) as a linear function with variables x u for u ∈ D k 1 ( ρ ′ ) with E h k 1 ( X ) = E f k 1 ( X ) = 0 . Then,

The first inequality follows from Proposition B.5. The second inequality follows from Jensen's inequality. Here we impose the third assumption on C 0 that

Let us make a remark that either by second property of f from Lemma D.1 when m = k 1 or by Lemma F.2 in the case when m>k 1 , we have the following: For w ≤ ρ ′ and m ≤ k ′ ≤ h( w ) ,

With this notation,

̸

We will estimate the three summands individually.

Part 3.1: Estimating first summand of (72) First, we apply Cauchy-Schwarz inequality,

Now, combining (70) and (68), we have

By setting we can conclude that

Part 3.2: Estimating second summand of (72) For the second summand of (72), we begin with the estimate for each u ∈ D k ( ρ ′ ) :

the derivation of (63) follows.

Part 3: Derivation of (64)

For w ≤ ρ ′ with m ≤ h( w ) ≤ k , let

̸

Since for each i ∈ [ d u ] we have f m,u i ( x ) = f m,u i ( x ≤ u i ) , we apply (39) from Proposition E.1 to ˜ f u and a ( x ) = f m,u i ( x ) to get

where C E. 1 is the constant introduced in the Proposition. Applying Jensen's inequality and (71) with w = u and k ′ = k -1 ,

Hence,

where

Now, returning to the summation, we apply (76) and Cauchy-Schwarz inequality to get

In the derivation above, we applied Lemma F.2 for the term ( ∑ u ∈ D k ( ρ ′ ) E ˜ f 2 u ( X ) ) 1 / 2 and (71) for the term ( ∑ u ∈ D k ( ρ ′ ) E f 2 m,u ( X ) ) 1 / 2 in the secont to last inequality. The constant C 2 in the last inequality is defined as

Part 3.3: Estimating third summand of (72) It remains to bound the third summand, it can be reduced to the upper bound for first summand. Applying the Cauchy-Schwarz inequality and Hölder's inequality we have

where in the last inequality we applied (74) and (71).

By setting

Now, combining the three estimates of the summands (75), (77), and (78) for (72) we conclude that

where

Now we impose the forth assumption on C 0 that

and (64) follows.

F.2 Proof of Theorem B.6

Let C 0 = C 0 ( M,d,c ∗ ) denote the constant introduced in the statement of the Lemma. Its precise value will be determined along the proof.

Let k 1 be a positive integer with the precise value to be determined later. Here we impose our first assumption on k 1 that

where C F. 1 is a constant that appears in Proposition F.1.

Now, let us consider a function f described in the Theorem. Without lose of generality, we may assume E f ( X ) = 0 . Then, it is equivalent to estimate the second moments.

Further, let us assume h( ρ ′ ) ≥ k 1 and the decomposition of f according to Lemma D.1:

̸

by showing E f k ( X ) f m ( X ) is insignificant whenever k = m .

For k 1 ≤ m<k , by Propostion F.1,

Applying the above inequality to bound the second moment of f ( X ) we get

Our first goal is to show

Notice that there exists t 0 which depends on ε so that

Our second goal is comparing E [ ( E ρ ′ f k ) 2 ( X ρ ′ ) ] and ∑ k ∈ [ k 1 , ∞ ] E f 2 k ( X ) . Starting with the variance and ℓ ∞ norm comparison from Lemma A.5,

where C A. 5 is the constant introduced in the Lemma.

By (62), from Proposition F.1, for k ∈ [ k 1 +1 , h( ρ ′ )] ,

Substituting these estimate and by Cauchy-Schwarz inequality we have

Now, by taking we conclude that

It remains to show the case when h( ρ ′ ) ≤ k 1 . From the assumption that C 0 ≥ C F. 1 + t 0 +1 and k 1 ≤ h ∗ + C F. 1 (log( R ) + 1) + t 0 +1 , we have

Hence, the statement follows directly from Jensen's inequality.

By setting we get

G General Case: Base Case

Now, we want to establish Theorem 1.6, which does not rely on the assumption c M > 0 . Let us first establish analogues of Assumption B.3 (the inductive assumption), Proposition B.5 (the base case), and Theorem B.1 (the inductive step) in the general case.

Assumption G.1. By stating that A satisfies this assumption with given parameter h ◦ , we mean A 1 ⊆ A ⊆ 2 L \{∅} is closed under decomposition, and the following holds:

For every u ∈ T and any A ≤ u -polynomials functions f and g , we have

Further, suppose E f = E g = 0 and h( u ) ≥ h ◦ . Notice that by the Markov Property, ( E u fg )( x ) , ( E u f 2 )( x ) , and ( E u g 2 )( x ) are functions of x u . Then,

The main difference of this assumption and Assumption B.3 is the difference of (82) and (16).

Proposition G.2. Consider the rooted tree T and transition matrix M described in Theorem 1.6. There exists C = C ( M,d ) ≥ 1 such that A 1 satisfies Assumption G.1 with some parameter h ◦ satisfying

Theorem G.3. Consider the rooted tree T and transition matrix M described in Theorem 1.6. There exists C = C ( M,d ) > 1 so that the following holds. Suppose A satisfies Assumption G.1 with some parameter h ◦ . Let B = B ( A ) (see Definition 1.11). Then, B satisfies Assumption G.1 with parameter

Proof of Theorem 1.6. The proof of Theorem 1.6 is analogous to that of Theorem B.1, employing a similar strategy by leveraging Proposition G.2 and Theorem G.3 in the former, and Proposition B.5 and Theorem B.6 in the latter.

In this section we will prove the Base Case Proposition G.2.

Lemma G.4. There exists a constant C = C ( M,ε ) ≥ 1 so such that for any ρ ′ ∈ T and 0 ≤ m ≤ h( ρ ′ ) :

Consider two degree 1 polynomials f and g with variables ( x u : u ∈ D m ( ρ ′ )) . Suppose

where f u ( x ) = f u ( x u ) and E [ f u ( X )] = 0 , and we assume the same conditions for the polynomial g and g u . Then,

Proof. Let C 0 = C 0 ( M,d ) denote the constant introduced in the statement of the Lemma. Its value will be determined along the proof.

First of all,

Our proof will be carried out by estimating each summand. Fix any pair u, v ∈ D m ( ρ ′ ) and consider

Let w = ρ ( u, v ) . Since f u and g v are functions of x ≤ w , relying on the Markov Property we know the function ( E w f u · g v )( x ) is a function of x w with expected value E f u ( X ) g v ( X ) . With

applying (10) from Lemma A.5,

where C A. 5 is the M -dependent constant introduced in the Lemma.

̸

Next, we will estimate ∥ ∥ ( E w f u g v )( θ ) -E f u g v ∥ ∥ ∞ . In the case u = v , there exists i = j such that u ≤ w i and v ≤ w j , which in turn implies that ( X ≤ u | X w = x w ) and ( X ≤ v | X w = x w ) are jointly independent by the Markov Property. Thus,

which implies

where we applied (10) and (9) from Lemma A.5 in the last inequality. If u = v , then the same estimate follows immediately without relying on (10).

Now, we convert the above estimate to that of ∥ ( E w f u g v )( θ ) -E f u g v ∥ ∞ , which relies on the simple bound that | E f u ( X ) g v ( X ) | ≤ max θ ∈ [ q ] | ( E w f u g v )( θ ) | . Thus,

Together we conclude that for a pair u, v ∈ D m ( ρ ′ ) with w = ρ ( u, v ) ,

Relying on this estimate, we are ready to bound the l ∞ -norm of ( E ρ ′ fg )( x ρ ′ ) -E fg .

Next, relaxing the condition ρ ( u, v ) = w in the summation,

̸

Notice the inequality ∑ i ∈ [ n ] | t i | n ≤ √ ∑ i ∈ [ n ] | t i | 2 n follows from Jenson's inequality applying to the function t ↦→ t 2 and the uniform measure on [ n ] . Now apply this inequality to the collection { √ E f 2 u ( X ) } and { √ E g 2 u ( X ) } respectively, together with | D m ( w ) | ≤ Rd h( w ) -m , from our tree asscumption, we have

where the last inequality follows from Cauchy-Schwarz inequality. Finally,

where

is a constant depending on M and ε . Combining the above estimate with (85) we conclude that

The statement of Lemma G.4 together with Proposition C.3 implies the following:

Corollary G.5. There exists a constant C = C ( M,ε ) ≥ 1 so that the following holds. For ρ ′ ∈ T and 0 ≤ m ≤ h( ρ ′ ) , consider two degree 1 polynomials f and g with variables ( x u : u ∈ D m ( ρ ′ )) with E f ( X ) = E g ( X ) = 0 . Notice that by the Markov Property, ( E ρ ′ fg )( x ) is a function of x ρ ′ . Then,

Remark G.6. By taking the degree 1 polynomial f = g with the assumption that E f ( X ) = 0 , we get

In other words, if h( ρ ′ ) is sufficiently large, ( E ρ ′ f 2 )( X ρ ′ ) is almost the same as E f 2 ( X ) with a small fluctuation. Let us state this as a seperate lemma.

Lemma G.7. There exists C = C ( M,d ) so that the following holds. For ρ ′ ∈ T with

any degree 1 polynomial f of variables ( x u : u ∈ L ρ ′ ) with E f ( X ) = 0 satisfies

Proof. By Corollary G.5, for every θ ∈ [ q ] ,

where C G. 5 is the constant introduced in Lemma G.5. Now, we set the constant described in the lemma as which implies

Then, with h( ρ ′ ) ≥ C (log( R ) + 1)

Proof of Proposition G.2. Let C 0 denote the constant introduced in the statement of the Proposition. Its precise value will be determined along the proof.

Let ρ ′ ∈ T with h ′ := h( ρ ′ ) . By Lemma C.9, any degree-1 polynomial f ( x ) with variables ( x u : u ∈ L ρ ′ ) satisfies

where C C. 9 denotes the M -dependent constant introduced in the Lemma. For the term in front of Var[ f ( X )] ,

where

Thus, if we impose the first assumption on C 0 that

then the first condition (81) in Assumption G.1 holds for A 1 if we take h ◦ ≥ C 0 (1 + log( R )) .

It remains to establish (82). Let f, g be two degree-1 polynomials in the variables ( x u : u ∈ L ρ ′ ) satisfying E f ( X ) = E g ( X ) = 0 . First, by Corollary G.5,

where C G. 5 is the constant introduced in Corollary G.5. Next, we would like to apply Lemma G.7. Assuming

which in term implies

where C G. 7 is the constant introduced in the Lemma, we can apply the lemma to get

and the same holds for g . Together we may conclude that

Now, we impose the second assumption on C 0 that

Then, we conclude that

provided that then, for every θ ∈ [ q ] ,

Therefore, we can conclude that A 1 satisfies Assumption G.1 with

H Inductive Step in General Case

The goal in this section is to prove Theorem G.3. Let us restate the theorem here:

Theorem. Consider the rooted tree T and transition matrix M described in Theorem 1.6. There exists C = C ( M,d ) > 1 so that the following holds. Suppose A satisfies Assumption G.1 with some parameter h ◦ . Let B = B ( A ) (see Definition 1.11). Then, B satisfies Assumption G.1 with parameter

In this section, we fix a subcollection A satisfying Assumption G.1 with a given parameter h ◦ and let B = B ( A ) .

We begin with the following lemma, which allows us to recycle some of the results from the case c M > 0 .

Lemma H.1. Suppose A satisfies Assumption G.1 with parameter h ◦ . Then, then A satisfies Assumption B.3 with h ∗ = h ◦ + 2 ε log(2) and c ∗ = 1 2 .

Proof. Let f be a A ≤ v -polynomial. If we set h ∗ ≥ h ◦ , then (15) follows immediately from (81).

Now, we assume that E f ( X ) = 0 and h( v ) ≥ h ◦ . We could apply (82) with g = f to get

If exp ( -ε 2 (h( v ) -h ◦ ) ) ≤ 1 2 , or equivalently,

Therefore, if we set h ∗ ≥ h ◦ + 2 ε log(2) and c ∗ = 1 2 , both (15) and (16) hold.

In the remainning of this section, we set

and we will rely on the fact that A satisfies Assumption B.3 with these two parameters. In particular, we could apply Theorem B.6 to show the existence of C B. 6 = C ( M,ε, 1 / 2) such that for any B ≤ v -polynomial f ,

Therefore, to establish Theorem G.3, it remains to show the existence of C = C ( M,d ) so that any B ≤ v -polynomials f and g with h( v ) ≥ h ◦ + C (log( R ) + 1) and E f ( X ) = E g ( X ) = 0 satisfy

To establish the above inequality, the higher level structure is essentially the same as that for deriving Theorem B.6. We again decompose f and g according to Lemma D.1. To the proof of the theorem, similarly it contains three steps:

Establish properties of ˜ f u and ˜ g u , see Proposition H.2.

Estalbish properties of f k and g k , see Proposition H.6.

Establish Theorem G.3.

H.1 Properties of f u

The main goal we want to prove in this subsection is the following Proposition.

Proposition H.2. There exsits C = C ( M,d ) ≥ 1 so that the following holds. For a given u ∈ T \ L with

suppose f u and g u are two functions which are linear combination of ψ σ ( x ) with σ ∈ F ( B u ) . Then, for any θ, θ ′ ∈ [ q ] ,

With a minor modification to our approach, we are able to obtain an analogous result wherein f u and g u are substituted by ˜ f u and ˜ g u , respectively:

Corollary H.3. There exsits C = C ( M,d ) ≥ 1 so that the following holds. For a given u ∈ T \ L with

suppose f u and g u are two functions which are linear combination of ψ S ( x ) with S ∈ F ( B u ) . Then, for any θ, θ ′ ∈ [ q ] ,

Let us prove Corollary first.

Proof. Let C 0 denote the constant introduced in the Corollary. Its value will be dervied during the proof.

From the identity

it follows that

First, we apply Propostion E.1 with the fact that A satisfies Assumption G.1 with parameter h ∗ = h ◦ + 2 ε log(2) and c ∗ = 1 2 ,

where C E. 1 = C ( M,d, 1 2 ) is the constant introduced in the Proposition.

Second, applying Proposition H.2 with f u = g u we have

where C H. 2 is the constant introduced in Proposition H.2.

Let us impose the first assumption that C 0 ≥ C H. 2 . Then, with h( u ) ≥ h ◦ + C 0 (log( R ) + 1) , we can conclude that

Clearly, the same derivation also holds for g u .

Therefore, we conclude that

where the last inequality follows by imposing the second assumption on C 0 that

This completes the proof of the Corollary.

The main technical part for proving Proposition H.2 is the following:

Lemma H.4. For any u ∈ T with exp ( -ε 2 (h( u ) -h ◦ ) ) ≤ 1 4 Rd , the following holds: Let I ⊂ [ d u ] be a subset of size at least 2. For any a ( x ) and b ( x ) which are linear combinations of ψ σ ( x ) with σ ∈ B u satisfying I ( σ ) = I , we have

Remark H.5. From the assumption that h( u ) satisfies

By taking a ( x ) = b ( x ) we have

Proof. Let u , a ( x ) , and b ( x ) be the vertex and functions described in the Lemma. Let us introduce some notations for the ease of expressing the calculation later. For brevity, let

For any given function h ( x ) with variables in ( x v : v ∈ ⋃ i ∈ I T u i ) , we define

Observe that

are functions with input x u,I . This is due to the fact that a and b -and consequently ab - are functions of variables ( x v : x v ∈ ⋃ i ∈ I L u i ) and Markov Property.

Claim: The function x u,I ↦→ ( E u,I ab )( x u,I ) is Lipschitz continuous with respect to the Hamming Distance with Lipschitz constant

We begin with the proof of the claim. Fix an index i 0 ∈ I . Without lose of generality, we assume I = [ k ] and i 0 = 1 . For x ∈ [ q ] T , let

With this notation above, we can express

Fix any value of x 0 , the function

For x ∈ [ q ] T , let for i ≤ [ d u ] , and set

is a linear combination of ˜ ϕ σ 1 ( x 1 ) with σ 1 ∈ F ( A ≤ u ) . Notably, this implies that E a ( x 0 , X 1 ) = 0 . The same properties hold for the function x 1 ↦→ b ( x 0 , x 1 ) .

Now, given the assumption exp ( -ε 2 (h( u ) -h ◦ ) ) ≤ 1 4 Rd implies h( u ) ≥ h ◦ , we can apply (82) from Assumption G.1 to get that

For any x ∈ [ q ] T , let x u 0 = ( x u 2 , x u 3 , . . . , x u du ) . By the Markov Property, for any θ ∈ [ q ] ,

are jointly independent. Hence,

where Y 0 is an independent copy of ( X 0 | X u 0 = x u 0 ) . We have

where the last inequality follows from Hölder's inequality. Further,

Applying the same derivation to b we get

Therefore, our claim (89) follows: For any θ, θ ′ ∈ [ q ] ,

With the Lipschitz continuity been established, essentially the lemma follows when δ is sufficiently small. Let us proceed with the remaining argument. Let x ′ u,I =argmin x u,I ( E u,I a 2 )( x u,I ) and x ′′ u,I =argmax x u,I ( E u,I a 2 )( x u,I ) . Applying (89) with the assumption a ( x ) = b ( x ) and the fact | I | ≤ d u , ( E u,I a 2 )( x ′′ u,I ) -( E u,I a 2 )( x ′ u,I ) ≤ 2 d u δ ( E u,I a 2 )( a ′′ u,I ) ,

and hence

provided that 2 d u δ < 1 .

Again, the same derivation also holds for b . Combining (89) and (90) we conclude that for any θ, θ ′ ∈ [ q ] ,

With our assumption on the tree T that d u ≤ Rd , our assumption implies that

We conclude that

Proof of Proposition H.2. Let C 0 = C 0 ( M,d ) denote the constant introduced in the statement of the Proposition. Recall the decomposition of f u into f u,I from Definition E.4, consider the decomposition

The proof of the Proposition will proceed by bounding summands in the formula below:

̸

Estimate of summands in (91): For any I, J ⊆ [ d u ] with | I | , | J | ≥ 2 , we have two cases to consider: First, we consider the case I = J . Notice that, by Lemma H.1, A satisfies Assumption B.3 with parameters (h ◦ + 2 ε log(2) , 1 2 ) . This allows us to invoke Corollary E.6, yielding

where C E. 6 = C E. 6 ( M,d, 1 2 ) is the constant introduced in the Corollary.

Let us impose the first assumption on C 0 that

which implies that h( u ) ≥ h ◦ + C 0 (log( R ) + 1) ≥ h ◦ + 2 ε log(4 dR ) . With this assumption, we could apply the remark (88) of Lemma H.4 to get

̸

and the same holds for g u,J . Hence, for I = J we have

Second, we consider the case I = J . Here we simply apply Lemma H.4, yielding

Let us unify the above two estimates by introducing

Then,

for every pair I, J ⊆ [ d u ] with | I | ≥ 2 and | J | ≥ 2 .

Using this inequality, (91) becomes

where ⃗ α = ( α I ) I ⊆ [ d u ] : | I |≥ 2 , ⃗ β = ( β ) I ⊆ [ d u ] : | I |≥ 2 , and A = ( a I,J ) I,J ⊆ [ d u ] : | I | , | J |≥ 2 . Further, ∥ ⃗ α ∥ and ∥ ⃗ β ∥ are the ℓ 2 norms of ⃗ α and ⃗ β , respectively, and ∥ A ∥ is the operator norm of A .

Estimate of operator norm of A : Notice that A is a symmetric matrix. Thus, we can fix a unit vector ⃗ γ satisfying ∥ A ∥ = ⃗ γ ⊤ A⃗ γ . For each pair I, J ⊆ [ d u ] with | I | ≥ 2 and | J | ≥ 2 , since a I,J ≥ 0 ,

and thus,

For each I ⊆ [ d u ] with | I | ≥ 2 , the number of J ⊆ [ d u ] with | I ∆ J | = k is bounded above by d k -1 u ≤ ( Rd ) k . Then, with the given estimate of a I,J in (92),

Now, we impose the second assumption on C 0 that

With the assumption that h( u ) ≥ h ◦ + C 0 (log( R ) + 1) , the geometric sum in (96) has a decay rate smaller than 1 / 2 . Therefore,

where

Now applying the above estimate, together with ∑ I ⊆ [ d u ] : | I |≥ 2 γ 2 I = 1 , to (95), we obtain the following bound:

Comparison of ∑ I min θ ∈ [ q ] ( E u f 2 u,I )( θ ) and min θ ∈ [ q ] ( E u f 2 u )( θ ) (and the same for g ): Here is the last step toward the proof of the Proposition. Returning to (94), we have

Let us impose the third assumption on C 0 that

where C E. 7 introduced in Corollary E.7. Recall that we have h ∗ = h ◦ + 2 ε log(2) from (87). We can invoke this Corollary to yield:

Let θ 0 ∈ [ q ] be the value minimizing θ ↦→ √ E f 2 u ( θ ) . Then,

Clearly, the same derivation also holds for g u . Together we conclude that

Finally, if we impose the forth assumption on C 0 that

then the Proposition follows.

H.2 Properties of f k : Products

The goal of this subsection is to establish the following.

Proposition H.6. There exists C = C ( M,d ) ≥ 1 so that the following holds. For any ρ ′ ∈ T satisfying

and a positive integer h ◦ + C (log( R ) + 1) ≤ k 1 ≤ h( ρ ′ ) . Consider a function f and g are B ≤ ρ ′ polynomials with E f ( X ) = E g ( X ) = 0 . We decompose f and g according to Lemma D.1 with the given k 1 . Then, the following holds: For k 1 ≤ m,k ≤ h( ρ ′ ) ,

If max { m,k } > k 1 ,

If k = m = k

The proof mirrors the structure used in Proposition F.1. In this case, we rely on both Proposition E.1 and Proposition H.2. Through this subsection, let

be the constant described in the Proposition. The functions f , g , and k 1 are as introduced in the Proposition.

Assuming without lose of generality that m ≤ k , we apply the reasoning from (72) in Proposition F.1, yielding

and hence,

Similar to the derivation of (64) from Proposition F.1. The proof is dedicated into estimating the above three summands.

We begin with the following estimate:

Lemma H.7. There exists a constant C = C ( M,d ) so that the following holds. Suppose C ◦ ≥ C H. 2 , where C H. 2 is the constant introduced in Proposition H.2. Then, the following holds: For u ∈ D k ( ρ ′ ) ,

if k > k 1 , then

where we applied (9) from Lemma A.5 with C A. 5 is the constant introduced in the Lemma. Further, relying on (98), together with (9) and (8) from the Lemma A.5, we have

where

Case 1: m<k . Here we can simply recycle the estimate from (76):

where

where C D. 1 is the constant introduced in Lemma D.1 and C E. 1 is the constant introduced in Proposition E.1.

Case 2: k 1 < m = k .

This is the case where we need Proposition H.2. With the assumption that C ◦ ≥ C H. 2 , where C H. 2 ≥ 1 is the constant introduced in the Proposition, we have

so that we could apply the Proposition to get

Case 3: k 1 = m = k The last case is straightforward:

By taking C 3 = max { C 2 , C H. 2 } + 2 ε log( C 1 ) , the statement of the Lemma follows with C = C 3 .

As an analogue of the above Lemma, we also have

Lemma H.8. There exists a constant C = C ( M,d ) so that the following holds. Suppose C ◦ ≥ C H. 2 is the constant introduced in Proposition H.2. Then, the following holds: For u ∈ D k ( ρ ′ ) ,

1. if k > k 1 , then

if k = m = k 1 , then

Since the proof is simpler and the structure is the same as that for Lemma H.7, we will outline a sketch proof in this case.

Proof. Let a u ( x u ) = ( E u ˜ f u )( x u ) and b u = ( E u g m,u )( x u ) . Repeating the first step of the proof of Lemma H.7, we have

with

which is exactly the same constant stated in Lemma H.7. Next,

If k > k 1 , we could apply (38) from Proposition E.1 to ˜ f u , and get

Indeed, this tail bound is stronger than what we got from Lemma H.7. The remainning part involves combining these estimates with a suitable constant C so that the lemma holds. Given the argument was already presented in the proof of Lemma H.7, we will omit these details.

Before bounding the summands in (97), let us bound ∑ u ∈ D k ( ρ ′ ) E ˜ f 2 u ( X ) and ∑ u ∈ D k ( ρ ′ ) E g 2 m,u ( X ) from above by E f 2 k ( X ) and E g 2 m ( X ) , respectively.

where C F. 2 is the constant introduced in Lemma F.2. Then,

and

Proof. Given that C ◦ ≥ C F. 2 + 2 ε log(2) , we have

and thus we could apply Lemma F.2. When m>k 1 , the lemma yields

And in the case when m = k 1 , we use the same derivation with Lemma F.2 been replaced by (31) in Lemma D.1 to get

Clearly, the same derivation also holds for the comparison of ∑ u ∈ D k ( ρ ′ ) E ˜ f 2 u ( X ) and E f 2 k ( X ) .

Now, relying on the above two lemmas, we will estimate the second and third summand of (97):

Corollary H.10. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds. Suppose

where the constants are introduced in Proposition H.2 and Lemma F.2, respectively. Then,

2. if k = m = k , then the above term above can be bounded by

Proof. Let C 1 be the maximum of the two constants introduced in Lemma H.7 and Lemma H.8. For convenience, let

By the two lemmas together with the triangle inequality, in the case when k > k 1 , we have

where the last inequality follows from Lemma H.9. Similarly, when k = m = k 1 , we have

By setting the corollary follows.

It remains to estimate the first summand of (97):

Lemma H.11. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds. Suppose

where C F. 2 is the constant introduced in Lemma F.2. Then,

1. if k > k 1 , then

2. if k = m = k 1 , then the above term is bounded by

Proof. Observe that both ( E k f k )( x ) = ∑ u ∈ D k ( ρ ′ ) ( E u ˜ f u )( x u ) and ( E k g m )( x ) = ∑ u ∈ D k ( ρ ′ ) ( E k g m,u )( x u ) are both degree-1 polynomials with variables ( x u : u ∈ D k ( ρ ′ )) satisfying

This allows us to apply Lemma G.4, yielding

where C G. 4 ≥ 1 is the constant introduced in Lemma G.4. Next, we apply Lemma H.9 (which is why we need the assumption on C ◦ ) to get

This leads to

If k > k 1 , then we can apply Proposition E.1 and Lemma H.9 to get

In this case, we have

2

C

G.

4

By taking

both statements of the lemma follows.

Proof of Proposition H.6. Without lose of generality, it is sufficient to prove the case when m ≤ k . First, we impose the first assumption that

{

4

, C

2

D.

R

4

}

exp

(

-

ε

(h(

ρ

′

)

-

C

(log(

R

) + 1)

1

E.

1

-

h

◦

-

2

ε

E

log(2))

f

2

k

(

X

)

√

E

g

2

m

(

X

)

.

where the constants are introduced in Proposition H.2 and Lemma F.2, respectively. This allows us to apply Corollary H.10 and Lemma H.11. For simplicity, let

Then, combining the Corollary and the Lemma to the estimate (97) we can conclude that: For k 1 ≤ m ≤ k with k > k 1 ,

and in the case where k = m = k 1 , the above term is bounded by

Then, the proof of the proposition follows by making the second assumption on C ◦ that

H.3 Proof of Theorem G.3

Proof. Now we are ready to establish the main theorem. As usual, let C 0 = C 0 ( M,d ) denote the constant introduced in the statement of the Theorem. The value of C 0 will be determined as the proof proceeds.

Applying Theorem B.6 with A and h ∗ = h ◦ + 2 ε log(2) and c ∗ = 1 / 2 , we conclude that

for any B ≤ ρ ′ -polynomial f , where C B. 6 = C ( M,d, 1 / 2) is the constant introduced by the theorem. We impose the first assumption on C 0 that

and conclude that

Now, it remains to show that with the suitable choice of C 0 , for any ρ ′ with h( ρ ′ ) ≥ h ◦ + C 0 (log( R )+ 1) and any two B ≤ ρ ′ -polynomials f and g , we have

Let where

t 0 is the constant such that ∑ ∞ t = t 0 exp ( -ε 2 t ) ≤ 1 2 ,

C H. 6 is the constant introduced in Proposition H.6, and

C F. 1 = C ( M,d, 1 / 2) is the constant introduced in Proposition F.1.

Next, let

The chocie of C 1 and k 1 allow us to apply Proposition H.6 and Proposition F.1 toward both f and g . Next, we impose the second assumption on C 0 that

This assumption implies that there is a gap between h ( ρ ′ ) and k 1 , which is necessary for the proof.

Now, we fix such ρ ′ and consider two B ≤ ρ ′ -polynomial f and g with E f ( X ) = E g ( X ) = 0 . Further, consider the decomposition of f and g according to Lemma D.1 with the above chosen k 1 .

First, by our choice of C 1 , we have

This assumption allow us to recycle the partial step in the proof of Theorem B.6 to obtain (80):

Second, with our assumption that k 1 ≥ ⌈ h ◦ + C H. 6 (log( R ) + 1) ⌉ , we can apply Proposition H.6 to get

Together with (103), we have

The next goal is to bound ∥ A ∥ from above. Notice the fact that the matrix A is symmetric implies there exists a unit vector ⃗ γ such that ∥ A ∥ = ⃗ γ ⊤ A⃗ γ . Now we fix such vector ⃗ γ . Relying on the fact that a km ≥ 0 ,

Clearly, from the definition of a km , the term ∑ k ∈ [ k 1 , h( ρ ′ )] a km is maximized when m = k 1 .

First,

Second,

which in turn implies that

Hence, we conclude that

Together we conclude that when h( ρ ) ≥ h ◦ + C 1 (log( R ) + 1) , any two B ≤ ρ ′ -polynomials f and g with E f ( X ) = E g ( X ) = 0 satisfies

Now, we impose the third assumption on C 0 that

then

Next, we apply (104) to the special case that f = g :

Clearly, the same statemnet holds for g as well. Substituting these estimates back to (104), we can conclude that when h( ρ ) ≥ h ◦ + C 0 (log( R ) + 1) , any two B ≤ ρ ′ -polynomials f and g with E f ( X ) = E g ( X ) = 0 satisfies

Therefore, the theorem follows.

I Variance Estimate for degree 1 polynomial

This section is dedicated to prove Proposition C.3. Let us restate it here:

Proposition I.1. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds: Fix ρ ′ ∈ T , and 0 ≤ k ≤ h( ρ ′ ) , then for any degree 1 function f with variables ( x u : u ∈ D k ( ρ ′ )) . There exists functions f u ( x ) = f u ( x u ) for u ∈ D k ( ρ ′ ) so that the following holds:

1. f ( X ) = ∑ u ∈ D k ( ρ ′ ) f u ( X u ) almost surely. (They may not agree as functions from [ q ] T to R .)

For any v ∈ T ρ ′ with h( u ) ≥ k ,

Example I.2. Suppose u, v ∈ c ( ρ ′ ) for u, v, ρ ′ ∈ T and consider

Let us consider the function f ( x ) = f u ( x ) + f v ( x ) where

and f v ( x ) = -1 1 , 3 ( x v ) .

Condition on X ρ ′ ∈ { 1 , 3 } , f ( X u ) + f ( X v ) = 1 -1 = 0 condition on X ρ ′ ∈ { 1 , 3 } and condition on X ρ ′ ∈ { 2 , 4 } , f ( X u ) + f ( X v ) = 0 -0 = 0 . Put it differntly, Var[ f ( X )] = 0 since f ( X ) = 0 almost surely. However, observe that π is the uniform measure on [4] , which implies

Therefore, it is not true that (23) holds for the standard (Efron-Stein) decomposition of f ( x ) = ∑ v ∈ D k ( ρ ′ ) f v ( x v ) .

Let us make a simple observation to give the insight for the construction. If f ( X u ) is a function of X p ( u ) , then for each i ∈ [ q ] , the function f must take the same value for all possible outcomes of X u conditioned on X p ( u ) = i . In other words, the values of f are constant on the set

for every i ∈ [ q ] . Now, let us consider the case where f ( X u ) is a function of X p r ( u ) . This can be reformulated as follows: for k ∈ [0 , r -1] , E [ f ( X u ) ∣ ∣ X p k ( u ) ] is a function of X p k +1 ( u ) . Equivalently, the values of M k f are constant on the set S i for every i ∈ [ q ] .

Therefore, it is evident that the construction of the basis should primarily revolve around the sets { S i } i ∈ [ q ] and their interaction with M .

Following from this discussion, the proof of the Proposition I.1 is divided into the following steps:

Step 1 (Section I.1): We try to give a precise description of when f ( X u ) is a function of X p k ( u ) for some k ∈ N . To this end, we introduce the following notation.

Definition I.3. We define the following partial order relation ≤ on the collection of all partitions of [ q ] : Specifically, for two partitions P and P ′ , we say that P ≤ P ′ if P ′ is finer than or equal to P .

Further, there exists r ∈ N such that P t, 0 for t ≥ r is the trivial partition.

Lemma I.4. There exists a chain of paritions

A function f : [ q ] ↦→ R satisfies that f ( X u ) is a function of X p r ( u ) for some r ∈ N if and only if f is a linear combination of 1 P for P ∈ P r, 0 .

(The double index for the partitions is due to a technical reason, which will be clear in the construction of the partitions.)

Step 2 (Section I.2): Next, we try to extract a basis of functions according to the partitions fro the previous step, along with suitable quantitative estimates:

Proposition I.5. Let M be an ergodic and irreducible transition matrix defined on the state space [ q ] . We can construct

a basis of functions from [ q ] to R , denoted as

and a constant C > 1 (which depends on M )

so that the following holds:

1. Let where W is a set of size q ,

a function

There exists unique w 0 ∈ W such that r ( w 0 ) = r 0 . Moreover, ξ w 0 ≡ 1 .

̸

For each w = w 0 , ξ w ( X u ) is a function of X v where v = p r ( w ) ( u ) and E ξ w ( X u ) = 0 .

3.

̸

For any 0 ≤ r ′ < r 0 such that { w ∈ W : r ( w ) = r ′ } is not empty,

For any 0 ≤ r ′ < r 0 such that { w ∈ W : r ( w ) < r ′ } is not empty,

Remark I.6. For w ∈ W and l ∈ [ r ( w )] , let

where we treated ξ as an vector in R [ q ] . Equivalently,

where u, v ∈ T are vertices such that v = p l ( u ) .

Step 3 (Section I.3): Finally, we will use the basis from the previous step to decompose degree-1 polynomials to prove Proposition I.1.

I.1 Partitions of [ q ]

Let us begin with the following observation.

Lemma I.7. Suppose { O α } α ∈ I is a collection of non-empty subsets of [ q ] . Then, there exists a unique partition P of [ q ] that satisfies the following 2 conditions:

For each α ∈ I and P ∈ P , either O α ∈ P or O α ∩ P = ∅ .

2. For any other partition P ′ that also satisfies the above property, P ′ ≤ P .

Proof. The proof can be carried out by constructing the partition P .

Without lose of generality, we may assume the collection { O α } α ∈ I contains { { θ } } θ ∈ [ q ] , since for a singleton { θ } and a set P , it is always true that either { θ } ⊆ P or { θ } ∩ P = ∅ . Consequently, we may assume

̸

First, we define an equivalence relation ≃ on { O α } α ∈ I as follows: For any α, α ′ ∈ I , we denote O α ≃ O α ′ if there exists a chain ( α 1 , α 2 , . . . , α l ) such that O α i -1 ∩ O α i = ∅ for i ∈ [ l ] . Let I 1 , . . . , I k 0 ⊆ I be the partition of I such that { O α } α ∈ I k for k ∈ [ k 0 ] form the equivalence classes of the relation. Now, let P := { P 1 , . . . , P k 0 } , where

Claim 1: For every α ∈ I and k ∈ [ k 0 ] , either O α ⊆ P k or O α ∩ P k = ∅ .

̸

To prove this claim, consider any α and k described above. Suppose O α ∩ P k = ∅ . Let θ ∈ O α ∩ P k and pick an index α ′ ∈ I k such that θ ∈ O α ′ . Such an index exists because P k = ⋃ α ′′ ∈ I k O α ′′ . Then, we have O α ∩ O α ′ = ∅ , implying α ∈ I k . Consequently, O α ⊆ P k . Therefore, the claim is proven.

Claim 2: P is a partition of [ q ] .

We need to verify three properties:

⋃ k ∈ [ k 0 ] P k = [ q ] ,

̸

∀ k ∈ [ k 0 ] , P k = ∅ , and

̸

P k ∩ P k ′ = ∅ whenever k = k ′ .

First, for each θ ∈ [ q ] , by (107), there exists α ∈ I such that θ ∈ O α . Then, θ ∈ O α ∈ P k where k is the index such that α ∈ I k . Hence, we conclude that ⋃ k ∈ [ k 0 ] P k = [ q ] .

Second, for each k ∈ [ k 0 ] , let α ∈ I k . We have ∅ ̸ = O α ⊆ P k . Thus, P k is not an empty set.

̸

Finally, for any distinct k, k ′ ∈ [ k 0 ] , suppose θ ∈ P k ∩ P k ′ . By (107), let α ∈ I be the index so that θ ∈ O α . Hence, both O α ∩ P k and O α ∩ P k ′ . In particular, it is necessary that α ∈ I k and α ∈ I k ′ , which forces k = k ′ , leading to a contradiction. Therefore, P k ∩ P k ′ = ∅ whenever k = k ′ . Hence, the claim follows.

Claim 3: P ′ ≤ P for any P ′ described in the statement.

To prove the claim, it suffices to show that for any P ′ ∈ P ′ and P k ∈ P with k ∈ [ k 0 ] , if P ′ ∩ P k = ∅ , then P k ⊆ P ′ .

̸

̸

Let us consider an arbitrary pair of P ′ ∈ P ′ and P k ∈ P and assume that P ′ ∩ P k = ∅ . There exists an index α such that O α ∩ P ′ ∩ P k = ∅ . Based on the assumptions regarding P and P ′ , we have α ∈ I k and O α ⊆ P ′ .

̸

̸

For every other α ′ ∈ I k , there exists a chain ( α = α 0 , α 1 , . . . , α l 0 = α ′ ) such that O α l -1 ∩ O α l = ∅ for l ∈ [ l 0 ] . Observe that if O α l -1 ⊆ P ′ , then O α l ⊆ P ′ , due to O α l ∩ P ′ ⊇ O α l ∩ O α l -1 = ∅ . With O 0 ⊆ P ′ as our starting point, we can apply this observation repeatedly to conclude that O α ′ ⊂ P ′ . Since the argument works for every α ′ ∈ I k , we conclude that P k = ⋃ α ′′ ∈ I k O α ′′ ⊆ P ′ .

̸

Definition I.8. For any given collection of subsets { O α } α ∈ I of [ q ] , let P ( { O α } α ∈ I ) denote the partition P defined in Lemma I.7.

̸

For any given partition Q of [ q ] , let

Remark I.9. Clearly, P SC ( Q ) ≤ Q .

Definition I.10. Let and

Let us remark that P 1 , 0 is the finest partition of [ q ] so that each part P ∈ P 1 , 0 either contains S i or disjoint from S i for i ∈ [ q ] .

We use double indices for indexing the partitions because constructing such a chain of partitions requires the creation of multiple partitions along the way, as we will illustrate shortly.

To proceed, let us begin with a simple observation.

Lemma I.11. If P ∈ P 1 , 0 , then

Suppose P 1 , 0 = { P 1 , P 2 , . . . , P k 0 } . Then, the collection Q := { Q 1 , Q 2 , . . . , Q k 0 } where

is also a partition provided that M is irreducible.

Proof. For i with S i ∩ P = ∅ , it is immediate that ( M 1 P ) i = 0 . Conversely, when S i ∩ P = ∅ , it is necessary that S i ⊆ P . Consequently, ( M 1 P ) i = ∑ j ∈ [ q ] M ij = 1 .

̸

To establish that Q is a partition, we need to demonstrate the following three conditions:

Q k ∩ Q k ′ = ∅ for all distinct k, k ′ ∈ [ k 0 ] .

⋃ k ∈ [ k ] Q k = [ q ] .

̸

Q k = ∅ for all k ∈ [ k 0 ] .

For the first condition, suppose there exists i ∈ Q k ∩ Q k ′ for some distinct k and k ′ . By definition, S i ⊆ P k and S i ⊆ P k ′ , which is a contradiction. Hence, Q k ∩ Q k ′ = ∅ .

For the second condition, for every i ∈ [ q ] , we know that S i ⊆ P k for some k . Consequently, i ∈ Q k , ensuring ⋃ α ∈ [ k ] Q k = [ q ] .

For the third condition, if we assume Q k = ∅ , implying that no i ∈ [ q ] satisfies S i ⊆ P k , then M is not irreducible, since the states in P k cannot be reached.

Definition I.12. Let P 1 , 1 = Q where Q is the partition described in Lemma I.11.

Lemma I.13. If P is a finite union of parts in P 1 , 0 , then

where Q is a finite union of parts in P 1 , 1 . The above map induces a bijection between subsets of [ q ] that are finite union of parts of P 1 , 0 and subsets of [ q ] that are finite union of parts of P 1 , 1 , in which preserve the inclusion relation is preserved.

Proof. Let us express P 1 , 0 = { P 1 , P 2 , . . . , P [ k 0 ] } and P 1 , 1 = { Q 1 , Q 2 , . . . , Q k 0 } where 1 Q k = M 1 P k .

For each I ⊆ [ k 0 ] , let P I = ⋃ k ∈ I P k and Q I = ⋃ k ∈ I Q k . Since 1 P I = ∑ k ∈ I 1 P k and 1 Q I = ∑ k ∈ I 1 Q k , clearly we have

Since naturally both finite union of parts of P and of Q are identified with a subset I ⊂ [ k 0 ] in the above way, the statement of the lemma follows.

where

An immediate consequence is the following.

Corollary I.14. The transition matrix M induces a bijection between partitions that are ≤ P 1 , 0 and partitions that are ≤ P 1 , 1 . For convenience, we adopt the following definitions:

Given any P ≤ P 1 , 0 and for each P ∈ P , let MP represent a part in M P where

.

Next, we will build a collection of partitions P r,s for r ≥ 0 and 0 ≤ s ≤ r starting with P 0 , 0 = { { 1 } , { 2 } , . . . , { q } } and establishing the relationship illustrated by the diagram below.

( In the above diagram, P → Q indicates that Q = M P ; Q ≥ SC P indicates P = P SC ( Q ) .)

Indeed, the initial definition of P 0 , 0 and the relation diagram determine the collection of partitions completely. Let us summarise it as a statement:

Lemma I.15. There exists a unique collection of partitions { P r,s } r ≥ s ≥ 0 that satisfies the following properties: For 0 ≤ s < r ,

Proof of Lemma I.15. The proof is proceeded by induction. We assume that P r,s is constructed and uniquely determined for 0 ≤ r < r 0 and 0 ≤ s ≤ r for some r 0 ≥ 0 so that it satisfies the properties described in the lemma.

We will define the partitions in the next column { P r 0 ,s } s ∈ [0 ,r 0 ] by starting with P r 0 ,r 0 -1 = P SC ( P r 0 -1 ,r 0 -1 ) .

Besides constructing the rest of partitions, we also need to show that these partitions satisfy the following list of conditions ( let us denote it as List A ): For s ∈ [0 , r 0 , -1] ,

By definition of the map P SC , the first and second condition in the list are satisfied for s = r 0 -1 . Relying on P r 0 ,r 0 -1 ≤ P 1 , 0 , we can define P r 0 ,r 0 = M P r 0 ,r 0 -1 . Hence, the third condition in the list is also satisfied for s = r 0 -1 .

It remains to construct P r 0 ,s for s ∈ [0 , r 0 -2] and they satisfy those 3 conditions in the list. This can be proceeded inductively starting from s = r 0 -2 .

Claim : For s ∈ [0 , r 0 -2] , if P r 0 ,s +1 ≤ P r 0 -1 ,s +1 , then there exists a unique partition P r 0 ,s which satisfies the conditions in List A for s .

Suppose the Claim holds. With P r 0 ,r 0 -1 ≤ P r 0 -1 ,r 0 -1 , we could apply the claim repeatedly and the lemma follows. The rest of the proof is to show the claim holds.

Let us assume P r 0 ,s +1 ≤ P r 0 -1 ,s +1 for some s ∈ [0 , r 0 -2] . First, from our assumption on { P r,s } for 0 ≤ s ≤ r 0 -1 , P r 0 -1 ,s +1 = M P r 0 -1 ,s . By Corollary I.14, P r 0 -1 ,s +1 ≤ P 1 , 1 . Since P r 0 ,s +1 ≤ P r 0 -1 ,s +1 , we conclude that P r 0 ,s +1 ≤ P 1 , 1 .

Applying Corollary I.14 again, we know there exists an unique partition P ≤ P 1 , 0 so that P r 0 ,s +1 = M P . We set P r 0 ,s := P . In particular, the choice of P r 0 ,s is unique in order to satisfy the first and third condition from the list.

It remains to show that P r 0 ,s also satisfies the second condition in List A . Notice that from Corollary I.14, the induced map of M on partitions preserves ≤ relation. Hence, P r 0 ,s +1 ≤ P r 0 -1 ,s +1 implies P r 0 -1 ,s ≤ P r 0 -1 ,s . Therefore, the claim holds.

Proof of Lemma I.4. We start with the proof on the ⇒ implication. Suppose f is a function satisfied the first condition described in the lemma.

Since f ( X u ) is a function of X p r ( u ) , this is equivalent to

Relying on the identity Var[ Y ] = E Var[ Y | Z ] + Var [ E [ Y | Z ] ] and ( X p r ( u ) , X p r -1 ( u ) , . . . , X u ) is a Markov Chain,

Hence, E [ Var [ f ( X u ) ∣ ∣ X p s ( u ) ] ] = 0 for s ∈ [ r ] , which in turn implies E [ f ( X u ) ∣ ∣ X p s -1 ( u ) ] conditioned on X p s ( u ) is a constant function for each s ∈ [ r ] . Equivalently, M s -1 f takes the same value for all elements in each S i for i ∈ [ q ] .

Claim: For s ∈ [ r ] , if f can expressed in the form f = ∑ P ∈ P s -1 , 0 c s -1 ,P 1 P , then it can be expressed in the form f = ∑ P ∈ P s, 0 c s,P 1 P .

Clearly, if the claim holds, then we can apply it repeatedly to draw the conclusion that f is a linear combination of 1 P for P ∈ P r, 0 .

Now, we fix s ∈ [ r ] and assume f = ∑ P ∈ P s -1 , 0 c s -1 ,P 1 P . Then,

where for each P ∈ P s -1 , 0 , P s -1 ∈ P s -1 ,s -1 is the corresponding part such that M s -1 1 P = 1 P s -1 . In other words, M s -1 f is a linear combination of 1 P for P ∈ P s -1 ,s -1 .

Because M s -1 f takes the same value not only for all elements in each S i for i ∈ [ q ] , but also for all elements in each P for P ∈ P s -1 ,s -1 , it implies M s -1 f takes the same value for all elements in each P ′ ∈ P SC ( P s -1 ,s -1 ) = P s,s -1 .

Together with the fact that the induced map of M on partitions preserves ≤ relation, we conclude that c s -1 ,P 1 = c s -1 ,P 2 for P 1 , P 2 ∈ P s -1 , 0 whenever P 1 and P 2 are both contained in some P ∈ P s, 0 .

Equivalently, within each P ∈ P s, 0 , f is a constant function. Hence, we can express f as a linear combination of 1 P for P ∈ P s, 0 .

For the ⇐ implication, suppose f is a linear combination of 1 P with P ∈ P r, 0 .

What we need to show is for s ∈ [0 , r -1] , M s f takes the same values for all elements in each S i for i ∈ [ q ] . From the chain P r, 0 → P r, 1 →··· → P r,r and by (108), for s ∈ [0 , r -1] , M s f is a linear combination of 1 P with P ∈ P r,s .

Since P s +1 ,s = P SC ( bP s,s ) ≤ P 1 , 0 and P s +1 ,s ≥ · · · ≥ P r,s , we have P r,s ≤ P 1 , 0 , which implies M s f takes the same values for all elements in each S i for i ∈ [ q ] . Therefore, the proof is completed.

Now, it remains to prove the second statement of the lemma.

First, if there exists r ∈ N such that P r, 0 is trivial. Then P t, 0 is also trivial for t > r since P t, 0 ≤ P r, 0 . Hence, it is enough to show the existence of r such that P r, 0 is trivial. P r, 0 is trivial.

From the assumption on M , we knew that the stationary distribution π of M satisfies min i ∈ [ q ] π ( i ) > 0 and M r converges entry-wise to the matrix whose row is identically π . Therefore, for sufficiently large r , min i,j ∈ [ q ] ( M r ) ij > 0 .

Now, let us fix such r and assume P r, 0 is not trivial. Let us express P r,s = { P r,s 1 , . . . , P r,s k r } with for s ∈ [0 , r ] and k r ≥ 2 where the index is assigned so that P r,s k = MP r,s -1 k for s ∈ [ r ] and k ∈ [ k r ] . First,

With 1 P r, 0 1 is non-negative and not zero, every component of M r 1 P r, 0 1 is non-zero. This forces P r,r 1 = [ q ] , which contradicts to the assumption that P r,r is non-trivial.

I.2 A basis of functions from [ q ] ↦→ R according to the partition

From now on, let r 0 be the smallest non-negative integer such that P r, 0 is trivial. Consider the collection

We will establish an identification between elements of the set described above and words whose alphabet consists of non-negative integers. This identification is constructed through induction, following these steps:

First, we identify ([ q ] , r 0 ) with the word (1) .

Assuming that elements in { ( P, s +1) : P ∈ P s +1 , 0 } have already been identified with unique words, we proceed as follows: For each ( P, s + 1) , suppose there are k pairs of ( P ′ , s ) such that P ′ ⊆ P . We identify these k pairs with the words ( w , i ) for i ∈ [0 , k -1] , in any order of preference. For each ( P ′ , s ) , due to P s, 0 is a finer than or equal to P s +1 , 0 , there exists an unique pair ( P, s +1) so that P ′ ⊆ P . This guarantees the above procedure assigns each ( P ′ , s ) a unique word.

We denote the set of words described above as ˜ W , and we adopt the notation w ∼ ( P, s ) to indicate that ( P, s ) is associated with the word w . For a given w ∈ ˜ W , we represent the corresponding pair as ( P w , r ( w )) , where r ( w ) = r 0 +1 -len( w ) .

Now, let us make the following observations

If w ∈ ˜ W is a word with len( w ) < r 0 +1 , then ( w , 0) ∈ ˜ W .

Each ( P, s ) corresponds to a word of length r 0 +1 -s .

Suppose w , w ′ ∈ ˜ W such that w is a prefix of w ′ . Then, P w ′ ⊆ P w .

Let T ˜ W be the tree defined on ˜ W using the prefix relation. In this tree, edges are drawn from w ′ to w if r ( w ′ ) = r ( w ) + 1 and P w ⊆ P w ′ . Now, we will select q parts from these elements ( P, s ) based on their corresponding words.

Lemma I.16. Let W ⊆ ˜ W be the subcollection of words which end with a positive integer. Then, | W | = q .

Proof. First of all, there are exactly q words in ˜ W with length r +1 , since P 0 , 0 = { { i } } i ∈ [ q ] has q parts. For each i ∈ [ q ] , let w ′ i be the word corresponding to ( { i } , 0) and let w i be the longest word ending with a positive integer so that is either a prefix of equals to w ′ i . This is well-defined since every word in ˜ W is a word starting with 1 .

The proof of the lemma follows if we can show the following claim : w 1 , w 2 , . . . , w q are distinct and are all words which ends with a positive integer.

̸

To prove the claim, we begin by showing w i = w j whenever i = j . Suppose w i = w j for some distinct pair of i, j ∈ [ q ] . Let ˜ w be the longest prefix of w ′ 1 , w ′ 2 , necessarily we have w i = w j is either a prefix of ˜ w or ˜ w itself. Further, the length of ˜ w is less equal than r , since otherwise it implies w ′ i = w ′ j , which is a contradiction.

̸

Now, let (˜ w , e i ) and (˜ w , e j ) be the two words which are prefix of w ′ i and w ′ j , respectively. From the definition that ˜ w is the longest common prefix, e i and e j are distinct non-negative integers. Since w i is a prefix of ( w , e i ) , it is necessary that e i = 0 , otherwise it violates the definition of w i . For the same reason, e j = 0 . Therefore, we reach a contradiction.

The remaining part to prove the claim is to show that { w i } i ∈ [ q ] are all the words in ˜ W ending with a positive integer. Suppose w is a word in which ends with a positive integer. If len ( w ) < r +1 , we can keep fill 0 until its length is r +1 and denote the resulting word by w ′ . Observe that w ′ ∈ ˜ W . Together with the length of w ′ is r +1 , necessarily w ′ = w ′ i for some i . Recall the definition of w i , we conclude w = w i . Therefore, the claim follows.

Lemma I.17. For any given 0 ≤ r ′ < r , suppose W r ′ := { w ∈ W : r ( w ) = r ′ } is non-empty. Consider a linear combination ∑ w ∈ W r ′ c w 1 P w . If it can be expressed as a linear combination of 1 P for P ∈ P r ′ +1 , 0 , then c w are identically 0 .

Proof. Let w 1 , . . . , w k 0 be the words with r ( w k ) = r ′ +1 and corresponding to each part of P r ′ +1 , 0 . Then, the words that corresponds to pairs of the form ( P, r ′ ) with P ∈ P r ′ , 0 are

where t k are non-negative integers. Now, we express

For each k ∈ [ k 0 ] and any θ ∈ P w k , we have

Therefore, ∑ w ∈ W r ′ c w 1 P w can be expressed as ∑ k ∈ [ k 0 ] c w k 1 P w k if and only if ∑ t ∈ [ t k ] c ( w , t ) 1 P ( w , t ) is a constant on P w k .

For each k ∈ [ k 0 ] , let θ ∈ P ( w k , 0) , then we have

which forces c = 0 for every t > 0 (if it exists). Therefore, the proof is complete.

( w k , t )

Definition I.18. Let B := { ξ w } w ∈ W be a collection of q functions from [ q ] to R , defined as follows:

If w = ( 1 ) , ξ w = 1 P w = 1 .

̸

If w = ( 1 ) ,

Remark I.19. The remaining goal in this subsection is to show that B is the desired basis described in Proposition I.5. We also remark that the first two properties stated in Proposition I.5 are already satisfied with this construction: argmax w ∈ W r ( w ) = ( 1 ) with ξ ( 1 ) = 1 [ q ] = 1 ; ξ w ( X u ) is a function of X v where v = p r ( w ) ( u ) .

Lemma I.20. The collection B forms a linear basis for functions from [ q ] to R .

Proof. Since there are exactly q functions, our goal is to show

and the R.H.S. is the same as span( { 1 P w } w ∈ W ) . It suffices to show for each i ∈ [ q ] , 1 { i } can be expressed as a linear combination of 1 P w with w ∈ W .

To prove this statement, we will use induction, showing that for s from r 0 to 0 , each 1 P with P ∈ P s, 0 can be expressed as a linear combination of of 1 P w with w ∈ W . Since P 0 , 0 = { { 1 } , . . . , { q } } , the proof follows once we establish this inductive statement.

First, when s = r , since 1 [ q ] is the only part in P r 0 , 0 and [ q ] = P ( 1 ) , the statement holds for s = r 0 .

Now, suppose the inductive hypothesis holds for s +1 with s < r 0 . Pick any P ∈ P s, 0 , let w = ( w ′ , t ) be the word associate with ( P, s ) . If t = 0 , then

where the sum is taken over all parts P ′′ ∈ P s, 0 \{ P } contained in P w ′ . Each P ′′ in the summation (if it exists) must corresponds to a word of the form ( w ′ , t ′′ ) with t ′′ > 0 , or equivalently ( w ′ , t ′′ ) ∈ W . From the induction hypothesis, 1 P w ′ is a linear combination of 1 P w with w ∈ W . Therefore, we conclude that 1 P is also a linear combination of 1 P w with w ∈ W . If t > 0 , then w ∈ W , and the same conclusion follows immediately. With no restriction on the choice of P , the induction hypothesis holds for s as well.

Therefore, the lemma follows from induction.

Lemma I.21. For any given 0 ≤ r ′ < r , suppose W r ′ := { w ∈ W : r ( w ) = r ′ } is non-empty. Then, there exists a constant C ≥ 1 (which could depends on M ) such that the following holds: Let u, v ∈ T be two vertices such that v = p r ′ ( u ) . We have

Proof. First, both sides of (109) scale by a factor h 2 if every term c w is multiplied by h ∈ R . Hence, it suffices to establish the inequality in the case

Given this, consider the set { ( c w ) w ∈ W r ′ : max w ∈ W r ′ | c w | = 1 } ⊆ R W r ′ . It is compact set and

is continuous in ( c w ) w ∈ W r ′ (it is a polynomial of c w ). By a compact argument one can estalbish the existence of C ≥ 1 described in the lemma if for every ( c w ) w ∈ W r ′ with max w ∈ W r ′ | c w | = 1 ,

We can simplify this by observing that

and hence,

where the second equality follows from that ∑ w ∈ W r ′ c w 1 P w ( X u ) is a function of X v by Lemma I.4.

is not a function of X p ( v ) . By Lemma I.4, this is equivalent to show ∑ w ∈ W r ′ c w 1 P w is not a linear combination of 1 P for P ∈ P r ′ +1 , which was proven in Lemma I.17. Therefore, the proof is complete.

Lemma I.22. For any given 0 ≤ r ′ < r , suppose W <r ′ := { w ∈ W : r ( w ) < r ′ } is non-empty. Then, there exists C ≥ 1 so that the following holds: Let u, v ∈ T be two nodes such that v = p r ′ ( u ) . We have

Proof. The proof is more straightforward compared to the arguments presented in the proof of Lemma I.21. First, both sides of (112) scale by a factor h 2 if we scaled each c w by h ∈ R . Therefore, it suffices to establish the inequality when

If there is no ( c w ) w ∈ W <r ′ satisfying the above condition, then the proof is completed. Now we assume this set is not empty. Notice that E Var [ E [ ∑ w ∈ W <r ′ c w ξ w ( X u ) ∣ ∣ X v ] ∣ ∣ ∣ X p ( v ) ] is a continuous function of ( c w ) w ∈ W <r ′ which takes value 0 when ( c w ) w ∈ W <r ′ = ⃗ 0 . Thus, there is an open ball B ⊆ R W <r ′ centered at ⃗ 0 such that for ( c w ) w ∈ W r ′ ∈ B ,

On the other hand, by choosing C sufficiently large, the set

which is the cube of side length 2 /C 1 / 2 centered at ⃗ 0 , is contained in B . Therefore, the lemma follows.

Proof of Proposition I.5. From Remark I.19 and Lemma I.20, it remains to show B satisfies the last 3 properties stated in the Proposition.

̸

As for the third property, notice that the variance of ∑ w ∈ W c w ξ w ( X u ) is not zero as long as c w are not identically 0 for w = w 0 . Following the same arguments in the proof of Lemma I.21, the property follows if C ≥ 1 is sufficiently large.

The last two follows by applying Lemma I.21 and Lemma I.22 to every 0 ≤ r ′ < r and choosing the constant C can be chosen to be the maximum of those constants C from the two lemmas.

I.3 Proof of Proposition I.1

In this subsection, we consider soley degree 1 polynomial of the leave values.

Definition I.23. For any given ρ ′ ∈ T and a degree-1 polynomial f of { x u } u ∈ L ρ ′ , the function can be expressed uniquely in the form

where { ξ w } w ∈ W is the basis introduced in Proposition I.5.

For u ∈ T ρ ′ , let f u ( x ) := ∑ w ∈ W , v ∈ L u c w ,v ξ w ( x v ) . Observe that from this definition, for each 0 ≤ l ≤ r ,

Remark I.24. From the definition above, for each ρ ′ ∈ T and degree-1 polynomial f of variables { x u } u ∈ L ρ ′ , we have

where ξ ( l ) w ( θ ) is introduced in Remark I.6.

Proposition I.25. There exists a constant C = C ( M,d ) ≥ 1 so that the following holds: Suppose

where ρ ′ ∈ T is a node satisfying h( ρ ′ ) ≤ r 0 and

for u, v ∈ L ρ ′ satisfying h( ρ ( u, v )) ≤ r ( w ) , where ρ ( u, v ) is the lowest common ancestor of u and v . Then,

If Proposition I.25 is proven, then Proposition I.1 follows as a corollary:

Proof of Proposition I.1. Reduction to h( ρ ′ ) ≤ r 0 : Without loss of generality, it is sufficient to consider degree 1 functions of L , rather than degree 1 functions of variables in D k ( u ) for some u in the tree and 0 ≤ k ≤ h( u ) .

Recall that

Further, for u ∈ T ρ ′ \ L ρ ′ , let

Weknowthat we can express f ( x ) = ∑ w ∈ D r 0 ( ρ ) f w ( x ) so that each of them is a degree-1 polynomial with variables { x u } u ∈ L w .

Together with the variance decomposition for degree-1 polynomials (See Lemma C.1)

it suffices to prove the same statement for degree-1 polynomials of x u with u ∈ L ρ ′ for ρ ′ satisfying h( ρ ′ ) ≤ r 0 .

Now, we fix such ρ ′ and consider

Averaging the Coefficients: For each w ∈ W and for each u ∈ D r ( w ) ( ρ ′ ) , we know that for any v 1 , v 2 ∈ L u ,

almost surely. As a consequcne, we have

almost surely. Now, we repeat this averaging process for each w ∈ W and for each u ∈ D r ( w ) ( ρ ′ ) . We denote the resulting function by ˜ f . While ˜ f and f may not be the same function, ˜ f ( X ) = f ( X ) almost surely. On the other hand, ˜ f is a function which satisfies the condition in Proposition I.25. Following from the proposition, we have

The proof is complete.

Let us begin with an intermediate step toward the proof of the Proposition I.25.

Lemma I.26. Suppose f is a function described in Definition I.23. For any given 1 ≤ l < r such that W l := { w ∈ W : r ( w ) = l } is non-empty. Let u ∈ T ρ ′ with h( u ) = r ( w ) , suppose

Then one of the following statement holds:

Either E Var [ ( E u f u )( X u ) | X p ( u ) ] ≥ π min 2 C 0 t 2 , or

max w ∈ W <l | c w ,u | ≥ √ π min 2 C 0 t .

Here, C 0 ≥ 1 is the constant C described in Proposition I.5 and π min := min θ ∈ [ q ] π ( θ ) .

Further, in the case when l = 0 , then we simply have E Var [ ( E u f u )( X u ) | X p ( u ) ] ≥ 1 C 0 t 2 .

Proof. We decompose ( E u f u )( x ) into three components:

where ξ ( l ) w is introduced in Remark I.6.

For each w with r ( w ) > l , ξ ( l ) w ( X u ) is a function of X v with v = p r ( w ) -l ( u ) . Hence, the last component ∑ w : r ( w ) >l c w,u ξ ( l ) w ( x u ) is a constant function whenever we condition on X p ( u ) . Consequently,

From Proposition I.5, we know that

where the constant C 0 is the constant C stated in the Proposition. Intuitively, from (115) it should be clear that if the R.H.S. of (114) is small, then E Var [ ∑ w : r ( w ) <l c w,u ξ ( l ) w ( X u ) ∣ ∣ ∣ X p ( u ) ] cannot be small. Let us derive this with a coarse estimate.

By (115), we know there exists θ ∈ [ q ] such that

Now, suppose Var [ ∑ w : r ( w ) <l c w,u ξ ( l ) w ( X u ) ∣ ∣ ∣ X p ( u ) = θ ] < t 2 4 C 0 . We could apply triangle inequality to get

and together with (114),

Consider the opposite case where Var [ ∑ w : r ( w ) <l c w,u ξ ( l ) w ( X u ) ∣ ∣ ∣ X p ( u ) = θ ] ≥ t 2 4 C 0 . First,

By applying the 4th property stated in Proposition I.5, we conclude that

In the case when l = 0 . The argument is simpler, which follows directly from (114) and the Proposition I.5.

Proof of Proposition I.25. Let t 0 = max w ,u ∈ L ρ ′ | c w ,u | and let w ′ ∈ W and u ′ ∈ L ρ ′ be the pair such that t 0 = | c w ′ ,u ′ | . Further, let l 0 = r ( w ′ ) and u 0 = p l ( u ′ ) . If l 0 > 0 , then we have

where the first equality follows from the assumptions of the coefficients. We will try to construct a sequence of triples ( l k , t k , u k ) indexed by k such that ( l k ) k ≥ 0 is strictly decreasing such that W l k = ∅ , h( u k ) = l k , and t k = max w ∈ W l k | c w ,u k | .

̸

̸

Suppose we have a triple ( l k , t k , u k ) such that l k ≥ 0 , h( u k ) = l k , W l k = ∅ , and t k = max w ∈ W l k | c w ,u k | for some index k ≥ 0 .

and set

If the first case is true, then we terminate the process of finding next triple ( ℓ k +1 , t k +1 , u k +1 ) . If the second case is true, let w ′′ ∈ W be the vertex such that | c w ′′ ,u k | = max w ∈ W <ℓ k | c w ,u k | ℓ k +1 = r ( w ′′ ) . Since

we have

Further, let u k +1 = argmax u ∈ T u k : h( u )= ℓ k +1 | c w ,u | .

In this way, we produce a new triple satisfying the same assumption as ( l k , t k , u k ) described above. Since l 0 > l 1 > l 2 . . . is a monotone decreasing chain of non-negative number, it means this argument must terminated in r 0 steps. Now, suppose it terminates at the k -th step, resulting a triple ( l k , t k , u k ) , and

Therefore, we conclude that

J Properties of Markov Chains and Galton-Watson Tree

J.0.1 Proof of Lemma A.5

Recall that real Jordon Canonical form of M is a q × q diagonal block matrix J = diag( J 0 , J 1 , . . . , J s 1 ) for some s 1 ≤ q .

Since M is ergodic, the eigenspace corresponds to eigenvalue 1 is 1-dimensional. Thus, We may assume J 0 = [1] is the unique Jordan block corresponds to eigenvalue 1 .

trix in R 2 with parameter θ s ∈ (0 , 2 π ) . In the later case, it corresponds to the conjugate pair of eigenvalues λ s (cos( θ s ) ± i sin( θ s ))

According to Jordon Decomposition, there exists an invertible matrix P such that M = P J P -1 .

For i ∈ [1 , q -1] , let ϕ i be the i +1 th column of P . Because P is invertible, { ϕ i } i ∈ [ q ] form a linear basis of functions from [ q ] to R .

Since π is a left-eigenvector of M with eigenvalue 1 , we have

because ϕ i is a sum of up to two generalized eigenvectors with eigenvalues not equal to 1 .

̸

(A generalized eigenvector v with eigenvalue λ ′ of M is a vector which satisfies ( M -λ ′ ) k v = ⃗ 0 for some positive integer k . Whenever λ ′ = 1 ,

If index i corresponds to J s which associated with a real eigenvalue, then ϕ i is a generalized eigenvector with eigenvalue λ s ; And if J s associates with a complex conjugate pair or eigenvalues, then ϕ i is a sum of two generalized eigenvectors with eigenvalues λ s (cos( θ s ) + i sin( θ s )) and λ s (cos( θ s ) -i sin( θ s )) , respectively. ) As a consequence, every function f : [ q ] ↦→ R can be uniquely decomposed in the form

With this unique decomposition, let us define a semi-norm

Lemma J.1. There exists C > 0 so that for every f : [ q ] → R ,

Proof. Without lose of generality, let f = ∑ i ∈ [2 ,q ] δ i ϕ i , since both ∥ f ∥ M and Var Y ∼ π ( f ( Y )) are invariant under a constant shift.

Let D π = diag( π 1 , . . . , π q ) . Also, let ⃗ δ = (0 , δ 2 , . . . , δ q ) . Then,

Let s max and s min be the maximum and minimum singular value of P ⊤ D π P , respectively. Together with q -1 / 2 ∥ ⃗ δ ∥ 2 ≤ ∥ ⃗ δ ∥ ∞ ≤ ∥ ⃗ δ ∥ 2 , we have

If s min > 0 , then we can complete the proof by taking C = max { s 2 max , q/s 2 min } . It remains to show that s min > 0 , or equvialently P ⊤ D π P is invertible. Because M is ergodic, each entry of π is positive, and thus D π is invertible. Hence, P ⊤ D π P is invertible because it is a product of three invertible matrices.

Lemma J.2. There exists C > 0 so that for every f : [ q ] → R ,

Proof. This simply follows from both ∥ f ∥ M and ∥ f -E f ∥ ∞ are both norms on the finite dimensional space { f : [ q ] → R : E Y ∼ π f ( Y ) = 0 } .

Lemma J.3. There exists C ≥ 1 depending on M such that For any function f : [ q ] ↦→ R and k ∈ N ,

Remark J.4. Notice that M k f can be interpreted as

for every u ∈ T where p k ( u ) is well-defined.

Proof.

Notice that J k is the diagonal block matrix whose blocks J k s for s ∈ [ s 1 ] . The block J k s can be computed directly: In the case when J s corresponds to a complex conjugate pair of eigenvalues,

where we treat ( k r ) = 0 if r > k . It can be verified directly by induction, relying on the identity ( k r -1 ) + ( k r ) = ( k +1 r ) . Further, removing the R s terms in the above equation we obtain the formula for J k s when J s corresponds to a real eigenvalue.

Therefore, with ( k q ) ≤ k q , | λ r s | ≤ λ r , and max i,j R r sij < 1 for r ≥ 1 , we obtain the bound

where C ′ is a constant which depends on q and λ .

Now we substitute the above bound into (122) to get

The proof is completed by taking C = qC ′ .

Proof of Lemma A.5. The proof of Lemma A.5 follows from the ∥ · ∥ M decay from Lemma J.3 and that both Var[ f ] and ∥ f -E f ∥ ∞ are comparable to ∥ f ∥ M within a constant multiplicative factor (Lemma J.1 and Lemma J.2).

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claim in the abstract and introduction is Theorem 1.6. The appendix is dedicated to the proof of the theorem.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The result is a theoretical result which justifies low degree hardness for the tree reconstruction model. This is a first indication that Kesten-Stigum bound appears to be the computational-statistical barrier for the problem. In terms of the limitation, the paper does not recover the same result as shown in the paper of Kohler and Moseel [23], where they only deal with the case λ = 0 .

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper provides a full set of assumptions and the appendix is dedicated to the complete proof for Theorem 1.6.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not include any experimental results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not include any experimental results.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not include any experimental results.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not include any experimental results.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not include any experimental results.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The result of the paper is a theoretical result, which suggests a previous known bound (Kesten-Stigum) is likely a statistical-computational bound for the tree reconstruction model. It is an generic result that suggest it is hard to design an efficient algorithm for the problem below the Kesten-Stigum bound. On the other hand, the result does not have any direct societal impact.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The result in the paper is a theoretical result. The paper does not process and realease any data or models.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study hardness of broadcasting on trees for low degree polynomials. The main result shows that log(N) degree polynomials of the leaf values have vanishing correlation with the root, resolving the main open question of Kohler and Mossel (NeurIPS 2022).\n\nThe tree broadcasting problem consists of a $d$-ary depth $\\ell$ tree T (or some relaxation thereof), an ergodic Markov chain $M$ over state space $[q]$, and a starting root distribution $\\nu$ over $[q]$. The broadcasting process starts by drawing $j \\sim \\nu$, then `broadcasts’ a value to every child in the tree based on the transition matrix $M$. This is repeated $\\ell$ times until the process terminates when it hits all leaves. Roughly speaking, the broadcasting problem asks whether it is possible to infer the starting value at the root given the values at the leaves of the tree given by the above process.\n\nThe most basic algorithm for tree broadcasting looks at linear statistics of the leaves, e.g. count statistics of how many times each state in [q] appears. There is a well-known threshold for linear algorithms called the Kesten-Stigum bound which states that such inference is possible if and only if $d\\lambda^2 < 1$, where $\\lambda$ is the 2nd largest eigenvalue of $M$. It is a natural (and known open) question whether this bound continues to hold against *low degree polynomials* of the leaves. This work answers this question in the positive for polynomials up to degree $\\log(|T|)$: all such functions have vanishing correlation with the root.\n\nSTRENGTHS:\nBroadcasting is a well studied and useful model. The question of how well low-degree statistics of the leaves correlate with the root is extremely natural mathematically, and is supported by a number of recent works showing the general power of low-degree polynomials in predicting statistical-computational gaps. The only known bounds for this general problem were for the very specialized $\\lambda=0$ case.\n\nThe techniques introduced by the authors, including the notion of fractal capacity (a more involved proxy for low degree functions that aids their proof), and the resulting inductive method for low-degree (low fractal capacity) functions may be of independent interest. As the authors note, it is one of the first methods for dealing with strongly non-product structures in this context, which is a very frequent barrier throughout related areas (e.g. analysis of Markov chains, boolean function analysis, etc).\n\nThe authors give a nice overview of their method in the symmetric linear case that is easy to follow (though a bit dense with the notions of fractal capacity introduced before this overview). Moving to the low-degree case requires highly non-trivial modifications of this method, but the intuition given there is very helpful.\n\nWEAKNESSES:\nI do not see any substantial weaknesses in this work. \n\nOne could of course ask for lower bounds against super-logarithmic degree polynomials as in known in the $\\lambda=0$ case, but the results in this paper still mark a major step forward on this problem.\n\nI would request the authors run a grammar/syntax check. There are a huge number of typographical/syntax errors that slow down the reading of the work to some extent.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-03"}

comments: SUMMARY:
The paper shows that for a Markov process in a tree, propagating a starting state at the root to the leaves it is "hard" to infer the starting state from the leaf states. 
Concretely they show that for any function $f$ of the leaf states with bounded Efron-Stein degree the variance of $f$ conditioned on the root state is bounded by a function of the total variance of $f$. That is, the output of such a function varies relatively little with respect to the choice of the root value which means that little can be inferred about the root value from the leaf values.
In particular since bounded degree polynomials have bounded Efron-Stein degree this implies that any low degree polynomial of the leaf values can not be correlated with the root, i.e. as tree depth goes to infinity the correlation goes to zero.This holds even for cases where it would be possible to recover some information about the root, so the result really does imply some bound on the power of low-degree functions.

STRENGTHS:
The paper shows a very nice result, and I feel like the overview in the main body gives a good idea of how the proof of the result proceeds.

WEAKNESSES:
There are no real weaknesses to the paper, however due to space constraints it is not really possible to ascertain the claimed results from the main body of the paper. I could not check the appendix carefully.

QUESTIONS:
- p2,l60 there is a broken reference
- On page 6 the application of the CS-inequality has a typo in the final term, it should be $i\in [m]$ below the sum
- p7,l203: of in a Markov Chain -> of a / in a 
- p8,l229 there is a missing ")" before the first square

LIMITATIONS:
No Limitations, this is theoretical research first and foremost
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper obtained low degree hardness results for broadcasting on trees (BOT). \nThe BOT problem is as follows. \nConsider a rooted tree and an associated Markov process defined by a transition matrix $M$. \nThe process is initialized by a distribution on the root. \nThen for each vertex with state $s$, its children are i.i.d. according to the $s$-th row of $M$. \nThe goal is to infer the state of the root given all leaves. \n\nThis paper shows that below the Kesten--Stigum threshold, any polynomial of the leaves of degree at most $\\log N$ fails to recover the root state. \nThis is particularly interesting since for some channels the KS threshold is information theoretically sharp and therefore they exhibit a stat-comp gap.\n\nSTRENGTHS:\nNA\n\nWEAKNESSES:\nNA\n\nQUESTIONS:\nI'm not familiar with this literature but I did try to check some of the cited papers to get a sense. \nThe result of this paper is certainly above the bar. \nHowever, there's (big) room for improvement of the presentation. \nIn fact, given the relevance of BOT and low-deg, upon proper revision of the exposition, the authors may want to send this paper to a journal instead. \n\nMajor comments: \n\n1. A related work section at the level of technique (i.e., low-deg) is needed. \nTo balance the space, the authors may want to reduce some parts of Sec 1.1 which are quite standard. \n\n1. Do the notions of $\\mathcal{B}(\\mathcal{A})$ and fractal capacity exist in the literature, or did the authors introduce them?\nIt's better to make it clear since the mere definitions of these are a nontrivial contribution IMO. \n\nMinor comments: \n\n1. Line 9, what do you mean by \"high level of complexity\"? It was just mentioned that BP runs in linear time.\n\n1. Line 10, at this point, it's not even clear what \"chain\" means. Please try to make the abstract self-contained. \n\n1. Line 20, based on. \n\n1. Line 22 doesn't parse grammatically. \n\n1. Line 22-25, please provide references. \n\n1. I don't understand line 37. Why compare low-deg **algorithms** to SoS **lower bounds**?\nAlso, what does \"easy to use\" mean?\n\n1. Line 40, [36] doesn't seem to involve AMP. Do you mean https://arxiv.org/abs/2212.06996 ?\n\n1. Line 42, \"broadcast on trees\" or \"broadcasting on trees\"? Please unify the terminology. \n\n1. Line 47, What do you mean by \"a linear estimator in the number of leaves\"?\nThe estimator is a linear function of the leaves? \nOr it runs in linear time as a function of the number of leaves?\n\n1. Line 54, our --> out\n\n1. Line 56, does \"sufficiently many states\" mean $q\\ge C$ for $C>5$? If so, it's better to make this clear. \n\n1. Line 60, broken reference. \n\n1. The discussion in line 54-61 is written in a chaotic way. \nPlease reorganize the relevant existing results. \n\n1. Line 68, leaves of for \n\n1. I don't understand the point of line 67-70.\nIs the message that for **small** $d$, large degree polynomials fail, but for **large** $d$, efficient reconstruction is possible?\nIn any case, these lines can be made more clear. \n\n1. Line 93, the notation $T$ hasn't even been introduced, so no need to abuse it. \n\n1. Line 96, begin with define\n\n1. Line 126, the notation $X_A = (X_v)_{v\\in A}$ hasn't been defined so far (correct me otherwise). \n\n1. Could the authors comment on the equation between line 139-140?\nIt says that conditioned on the root, the variance of (a low degree function of) the leaves will drop drastically (exponentially in depth). \nThis seems to imply that $f(X_L)$ is highly correlated with $X_\\rho$ and the correlation is **increasing** with the depth. \nApparently my interpretation is very wrong and contradicts the main message of the paper. \nCould the authors remark on why this is the correct statement to prove and how this implies Corollary 1.8?\n\n1. Corollary 1.8: Please define the correlation $\\mathrm{Cor}$. \n\n1. Line 146, \"the main result is optimal in the fractal sense\". This is interesting. \nCould the authors expand on this (maybe after the fractal capacity and stuff are properly defined later)?\n\n1. Line 154, the font of $b_1, \\dots, b_k$ changed. \n\n1. Definition 1.11, to make sure I understand it correctly, $\\mathcal{B}(\\mathcal{A})$ is **not** the closure (under decomposition) of $\\mathcal{A}$, right?\n\n1. Line 171, for $i$ for $i$\n\n1. Line 178, redundant line between end of proof and $\\square$. \n\n1. Definition 1.14, an $\\mathcal{A}$-polynomial\n\n1. Line 193, $2$-ary --> binary\n\n1. Line 194, eigenvalue --> eigenvalues\n\n1. Line 199, including in cases\n\n1. End of page 6, comparing to --> compared to\n\n1. The last equation of page 6 is unnecessary. \n\n1. Equation in line 202, is last $\\lesssim$ simply $=$?\n\n1. I(1), what is $S$?\n\n1. I(2), what is $S'$? In fact $S'$ is not even used in I(2). \n\n1. Line 229, What is $X$? I don't think $X$ is the whole tree?\nAlso, there's a missing right parenthesis. \n\n1. Line 230, satisfies --> satisfy. \n\n1. Line 234, \"builds on this strategy\", which strategy? This sentence feels out of place. \n\n1. Equation above line 242, what is $\\mathcal{J}$?\n\n1. Line 242, whose variables is --> are\n\n1. Below line 244, $x_{x_{\\le w_1}}$. \n\n1. Line 252, will also holds --> hold\n\n1. Equation above line 262, the argument of the function is included on the LHS but not on the RHS of the equation. \n\n1. Somewhere near the end of page 9, the font of $h_{\\mathcal{A}_k}$ changed.\n\n1. Not that it matters, but I don't think the authors used the latest version of the NeurIPS template which has a more tedious checklist.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-14"}

comments: SUMMARY:
This submission considers the broadcasting on trees problem, where given a rooted tree, information is propagated from the root to the leaves using a Markov process. The algorithmic task is to infer the information at the root given only the information at the leaves. Previous works had identified that the KS-threshold (a threshold based on the spectral gap of the Markov process and the degree-structure of the tree) is the critical threshold when this is possible information-theoretically in some specific cases. However, in general it is not the right information-theoretic threshold, i.e., in some cases inference is possible even below it. The submission gives evidence that it is indeed the right threshold however when considering computationally bounded algorithms. In particular, it establishes hardness in the low-degree framework below the KS-threshold.

STRENGTHS:
Previous work had asked the question whether the KS-threshold is the correct threshold for efficient algorithms. This work answers this question in the affirmative. The ideas used in the proof are novel and clever.

WEAKNESSES:
The first part of the introduction is a bit hard to follow for non-experts, maybe some additional context would help.

QUESTIONS:
No questions.

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-14