Lattice partition recovery with dyadic CART

1 Introduction . Suppose we observe a noisy realization of a structured , piece-wise constant signal supported over a d-dimensional square lattice ( or grid graph ) Ld , n = { 1 , . . . , n } d. Data that can be modeled in this manner arise in several application areas , including in satellite imagery [ e.g . 30 , 37 ] , computer vision [ e.g . 6 , 38 ] , medical imaging [ e.g . 26 , 23 ] , and neuroscience [ e.g . 16 , 32 ] . Our goal is to estimate the constancy regions of the underlying signal . Specifically , we assume that the data y ∈ RLd , n are such that , for each coordinate i ∈ Ld , n , yi = θ ∗ i + i , ( 1 ) where ( i , i ∈ Ld , n ) are i.i.d . N ( 0 , σ2 ) noise variables and the unknown signal θ∗ ∈ RLd , n is assumed to be piece-wise constant over an unknown rectangular partition of Ld , n . We define a subset R ⊂ Ld , n to be a rectangle if R = ∏d i=1 [ ai , bi ] , where [ a , b ] = { j ∈ Z : a ≤ j ≤ b } , a , b ∈ Z . A rectangular partition of Ld , n , P , is a collection of disjoint rectangles { Rl } ⊂ Ld , n , satisfying ∪R∈PR = Ld , n . To each vector in Ld , n , there corresponds a ( possibly trivial ) rectangular partition . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Definition 1 A rectangular partition associated with a vector θ ∈ RLd , n is a rectangular partition { Rl } l∈ [ 1 , k ] of Ld , n , such that θ takes on constant values over each Rl . For a vector θ ∈ RLd , n , we let k ( θ ) be the smallest positive integer such that there exists a rectangular partition with k ( θ ) elements and associated with θ . In this paper , we are interested in recovering a rectangular partition associated with the signal θ∗ in ( 1 ) . A complication immediately arises when d ≥ 2 : the rectangular partition associated with a given θ is not necessarily unique . This fact is illustrated in Figure 1 , where the left-most plot depicts the lattice supported vector θ , consisting of a rectangle of elevated value ( in grey ) against a background ( in white ) . For such θ , we show three possible rectangular partitions , each of which comprised of five rectangles ( the second , third and fourth plots ) . In fact , the partition recovery problem is well defined , as long as we consider coarser partitions comprised by unions of adjacent rectangles instead of individual rectangles : see Definition 2 below for details . We remark that this issue does not occur in the univariate ( d = 1 ) case , for which the partition recovery task has been thoroughly studied in the change point literature ; see section 1.3 below . Thus , we assume that d ≥ 2 . For the purpose of estimating the rectangular partition associated with θ∗ ( or , more precisely , its unique coarsening as formalized in Definition 2 ) , we resort to the dyadic classification and regression tree ( DCART ) algorithm of [ 14 ] . This is a polynomial-time decision-tree-based algorithm developed for de-noising purposes for signals over lattices , and is a variant of the classification and regression trees ( CART ) [ 7 ] . See Section 1.1 below for a description of DCART . The optimal regression trees ( ORT ) estimator , recently proposed in [ 12 ] , further builds upon DCART and delivers sharp theoretical guarantees for signal estimation while retaining good computational properties – though we should mention that in our experiments we have found DCART to be significantly faster . Both DCART and its more sophisticated version ORT can be seen as approximations to the NP-hard estimator θ1 = arg minθ∈RLd , n { 2−1‖y − θ‖2 + λk ( θ ) } , ( 2 ) where k ( θ ) is given in Definition 1 , ‖ · ‖ is the vector ( or Euclidean ) ` 2-norm and λ > 0 a tuning parameter . DCART modifies the above , impractical optimization problem by restricting only to dyadic rectangular partitions . This leads to significant gains in computational efficiency without sacrificing on the statistical performance . Indeed , decision-tree-based algorithms have been shown to be optimal under various settings for the purpose of signal estimation ; see [ 12 , 20 ] . In this paper , we further demonstrate their effectiveness for the different task of partition recovery . In particular , we show how simple modifications of the DCART ( or ORT ) estimator yield practicable procedures for partition recovery with good theoretical guarantees and derive novel localization rates . Note that , there is a wide array of applications focusing on detecting the regions rather than estimating the background signals , especially in surveillance and environment monitoring . Our work is motivated by all the applications/problems considered in the large literature on biclustering , where the underlying signal is assumed to be piecewise constant . Estimating the boundary of the partition is the most refined and difficult task in these settings . Thus , any of the many scenarios in which biclustering is relevant can be used to motive our task . An analogous observation holds also for the more general problem of identifying an anomalous cluster ( sub-graph ) in a network , a problem that has been tackled ( for testing purposes only ) by [ 4 ] , the reference therein provide numerous examples of applications . On a high-level , the relationship between the partition and signal recoveries can be thought of the relationship between the estimation consistency and support consistency in a high-dimensional linear regression problems . They can be done by almost identical algorithms but the theoretical results rely on different sets of conditions . The paper is organized as follows . In the rest of this section we formalize the problem settings and the task of partition recovery , and describe the DCART procedure . We further summarize our main findings and discuss related literature . Section 2 contains our main results about one- and two-sided consistency of DCART and its modification . In Section 2.3 we derive a minimax lower bound stemming from the case of one rectangular region of elevated signal against pure background . Illustrative simulations corroborating our findings can be found in Section 3 . The Supplementary Material contains the proofs . Notation We set N = nd , the size of the lattice Ld , n , where we recall that d ≥ 2 is assumed fixed throughout . For any integer m ∈ N∗ , let [ m ] = [ 1 , m ] . Given a rectangular partition Π of Ld , n , let S ( Π ) be the linear subspace of RLd , n consisting of vectors with constant values on each rectangle in Π and let OS ( Π ) ( · ) be the orthogonal projection onto S ( Π ) . For any R ⊂ Ld , n and θ ∈ RLd , n , let θ̄R = |R|−1 ∑ i∈R θi , where | · | is the cardinality of a set . Two rectangles R1 , R2 ∈ Π are said to be adjacent if there exists l ∈ [ d ] such that R1 and R2 share a boundary along el and one is a subset of the other in the hyperplane defined by el , the lth standard basis vector in Rd . See Definition S1 for a rigorous definition . This concept of adjacency is specifically tailored to – and in fact only valid for – dyadic ( and hierarchical , in the sense specified by [ 12 ] ) rectangular partitions , which are most relevant for this paper . For any subsets A , B ⊂ Ld , n , define dist ( A , B ) = mina∈A , b∈B ‖a − b‖ . Throughout this paper , we will use the ` 2-norm as the vector norm . 1.1 Problem setup . We begin by introducing two key parameters for the model specified in ( 1 ) and a well-defined notion of rectangular partition induced by θ∗ . Definition 2 ( Model parameters , induced partitions ) Let θ∗ as in ( 1 ) and { R∗j } j∈ [ m ] be a rectangular partition of Ld , n associated with θ∗ . Consider the graphG∗ = ( E∗ , V ∗ ) , where V ∗ = [ m ] and E∗ = { ( i , j ) : θ̄∗R∗i = θ̄ ∗ R∗j , R∗i and R ∗ j are adjacent } . Let { C∗l } l∈ [ L ] be all connected components of G∗ and define Λ∗ = Λ∗ ( θ∗ ) = { ∪j∈C∗1R ∗ j , . . . , ∪j∈C∗LR ∗ j } as the partition ( not necessarily rectangular ) induced by θ∗ . We say that the union of rectangles ∪j∈C∗sR ∗ j and ∪j∈C∗t R ∗ j , s , t ∈ [ L ] , s 6= t , are adjacent , if and only if there exists ( i , j ) ∈ C∗s × C∗t such that R∗i and R∗j are adjacent . Let κ and ∆ be the minimum jump size and minimal rectangle size , respectively , formally defined as κ = min a∈A , b∈B , A , B∈Λ∗ , θ∗a 6=θ∗b |θ∗a − θ∗b | and ∆ = min j∈ [ m ] |R∗j | . It is important to emphasize the difference between a partition associated with θ∗ , as described in Definition 1 , which may not be unique , and the partition Λ∗ induced by θ∗ of Definition 2 , which is instead unique and thus describes a well-defined functional of θ∗ . The parameters κ and ∆ capture two complementary aspects of the intrinsic difficulty of the problem of estimating Λ∗ ; intuitively , one would expect the partition recovery task to be more difficult when κ and ∆ are small ( and σ is large ) . Below , we will prove rigorously that this intuition is indeed correct . When d = 1 , both parameters , along with σ , have in fact been shown to fully characterize the change point localization task : see , e.g. , [ 36 , 35 ] . The partition recovery task can therefore be formulated as that of constructing an estimator Λ̂ of Λ∗ , the induced partition of θ∗ , such that , as the sample size N grows unbounded and with probability tending to one , |Λ̂| = |Λ∗| and ∆−1dHaus ( Λ̂ , Λ∗ ) = ∆−1 max A∈Λ∗ min B∈Λ̂ |A4B| → 0 , ( 3 ) where A4B is the symmetric difference between A and B . We refer to dHaus ( Λ̂ , Λ∗ ) as the localization error for the partition recovery problem . The dyadic classification and regression trees ( DCART ) estimator . In order to produce a computationally efficient estimator of Λ∗ satisfying the consistency requirements ( 3 ) , we deploy the DCART procedure [ 14 ] , which can be viewed as an approximate solution to the problem in ( 2 ) . Instead of optimizing over all vectors in RLd , n , DCART minimizes the objective function only over vectors associated with a dyadic rectangular partition , which is defined as follows . LetR = ∏ i∈ [ d ] [ ai , bi ] ⊂ Ld , n be a rectangle . A dyadic split of R chooses a coordinate j ∈ [ d ] , l the middle point of [ aj , bj ] , and splits R into R1 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ aj , l ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] and R2 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ l+1 , bj ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] , with [ 0 ] = [ c2 , c1 ] = ∅ , c2 > c1 . Assuming that n is a power of 2 , starting from Ld , n itself , we proceed iteratively as follows . Given the partition { Ru } u∈ [ k ] , one chooses a rectangle Ru and performs a dyadic split on Ru that leads to the largest reduction in the objective function . Any partition constructed through a sequence of such steps is called a dyadic rectangular partition . With a pre-specified λ > 0 , the DCART estimator is θ̃ = OS ( Π̃ ) ( y ) , where Π̃ ∈ arg min Π∈Pdyadic , d , n { 2−1‖y −OS ( Π ) ( y ) ‖2 + λ|Π| } , ( 4 ) where Pdyadic , d , n is the set of all dyadic rectangular partitions of Ld , n . As shown in [ 14 ] and [ 12 ] , the DCART estimator can be obtained via dynamic programming with a computational cost of O ( N ) . Given any solution to ( 4 ) , a natural ( though , as we will see , sub-optimal ) estimator of the induced partition of θ∗ is Π̃ , the partition associated with the resulting DCART estimator θ̃ . Importantly , by the property of DCART , and using the fact that the Gaussian errors have a Lebesgue density , Π̃ is in fact a dyadic-rectangular partition and is unique with probability one , and thus the resulting estimator is well-defined . ( Equivalently , the partition associated with θ̃ and the one induced by θ̃ coincide . )

Summary: The authors consider the problem of recovering the partition underlying a piecewise constant function over a $d$-dimensional lattice, _i.e._, a signal with support $L_{d,n} = \lbrace 1, \dots, n\rbrace^d$. That is, we wish to recover the structure such that within each rectangular partition of the lattice, the function $\theta^* \in \mathbb{R}^{L_{d,n}}$ is piecewise constant, from noisy measurements ${y_i = \theta_{i}^* + \epsilon_i}$ for $i \in L_{d,n}$ where $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$ i.i.d. A straightforward formulation would be NP-hard, so they the consider the dyadic CART (DCART) algorithm, which simplifies the search space by restricting it to partitions that can be formed by repeated bisections of partitions, starting with a single partition containing all $N = n^d$ points in the lattice.  The authors provide a number of bounds of the form $\sigma^2 k_{\mathrm{dyad}}(\theta^*) \log(N) / \kappa^2$, where $\kappa$ is the minimum jump size (difference between signal in adjacent partitions) of $\theta^*$. First, they show that using DCART, the number of lattice points in recovered partitions where the true signal is not actually piecewise constant scales at most with the above expression ("one-sided consistency"). They then propose a two-step estimator that adds a step merging rectangles in recovered partitions whose measurements are "close" to each other, and show that under certain assumptions, the Hausdorff distance between the recovered (estimated) partition and the actual partition (accounting for the need to merge rectangles to assure definitional uniqueness) scales at most with the above expression. Finally, they provide a simple example to demonstrate an order-matching lower bound.