Lattice partition recovery with dyadic CART

1 Introduction . Suppose we observe a noisy realization of a structured , piece-wise constant signal supported over a d-dimensional square lattice ( or grid graph ) Ld , n = { 1 , . . . , n } d. Data that can be modeled in this manner arise in several application areas , including in satellite imagery [ e.g . 30 , 37 ] , computer vision [ e.g . 6 , 38 ] , medical imaging [ e.g . 26 , 23 ] , and neuroscience [ e.g . 16 , 32 ] . Our goal is to estimate the constancy regions of the underlying signal . Specifically , we assume that the data y ∈ RLd , n are such that , for each coordinate i ∈ Ld , n , yi = θ ∗ i + i , ( 1 ) where ( i , i ∈ Ld , n ) are i.i.d . N ( 0 , σ2 ) noise variables and the unknown signal θ∗ ∈ RLd , n is assumed to be piece-wise constant over an unknown rectangular partition of Ld , n . We define a subset R ⊂ Ld , n to be a rectangle if R = ∏d i=1 [ ai , bi ] , where [ a , b ] = { j ∈ Z : a ≤ j ≤ b } , a , b ∈ Z . A rectangular partition of Ld , n , P , is a collection of disjoint rectangles { Rl } ⊂ Ld , n , satisfying ∪R∈PR = Ld , n . To each vector in Ld , n , there corresponds a ( possibly trivial ) rectangular partition . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Definition 1 A rectangular partition associated with a vector θ ∈ RLd , n is a rectangular partition { Rl } l∈ [ 1 , k ] of Ld , n , such that θ takes on constant values over each Rl . For a vector θ ∈ RLd , n , we let k ( θ ) be the smallest positive integer such that there exists a rectangular partition with k ( θ ) elements and associated with θ . In this paper , we are interested in recovering a rectangular partition associated with the signal θ∗ in ( 1 ) . A complication immediately arises when d ≥ 2 : the rectangular partition associated with a given θ is not necessarily unique . This fact is illustrated in Figure 1 , where the left-most plot depicts the lattice supported vector θ , consisting of a rectangle of elevated value ( in grey ) against a background ( in white ) . For such θ , we show three possible rectangular partitions , each of which comprised of five rectangles ( the second , third and fourth plots ) . In fact , the partition recovery problem is well defined , as long as we consider coarser partitions comprised by unions of adjacent rectangles instead of individual rectangles : see Definition 2 below for details . We remark that this issue does not occur in the univariate ( d = 1 ) case , for which the partition recovery task has been thoroughly studied in the change point literature ; see section 1.3 below . Thus , we assume that d ≥ 2 . For the purpose of estimating the rectangular partition associated with θ∗ ( or , more precisely , its unique coarsening as formalized in Definition 2 ) , we resort to the dyadic classification and regression tree ( DCART ) algorithm of [ 14 ] . This is a polynomial-time decision-tree-based algorithm developed for de-noising purposes for signals over lattices , and is a variant of the classification and regression trees ( CART ) [ 7 ] . See Section 1.1 below for a description of DCART . The optimal regression trees ( ORT ) estimator , recently proposed in [ 12 ] , further builds upon DCART and delivers sharp theoretical guarantees for signal estimation while retaining good computational properties – though we should mention that in our experiments we have found DCART to be significantly faster . Both DCART and its more sophisticated version ORT can be seen as approximations to the NP-hard estimator θ1 = arg minθ∈RLd , n { 2−1‖y − θ‖2 + λk ( θ ) } , ( 2 ) where k ( θ ) is given in Definition 1 , ‖ · ‖ is the vector ( or Euclidean ) ` 2-norm and λ > 0 a tuning parameter . DCART modifies the above , impractical optimization problem by restricting only to dyadic rectangular partitions . This leads to significant gains in computational efficiency without sacrificing on the statistical performance . Indeed , decision-tree-based algorithms have been shown to be optimal under various settings for the purpose of signal estimation ; see [ 12 , 20 ] . In this paper , we further demonstrate their effectiveness for the different task of partition recovery . In particular , we show how simple modifications of the DCART ( or ORT ) estimator yield practicable procedures for partition recovery with good theoretical guarantees and derive novel localization rates . Note that , there is a wide array of applications focusing on detecting the regions rather than estimating the background signals , especially in surveillance and environment monitoring . Our work is motivated by all the applications/problems considered in the large literature on biclustering , where the underlying signal is assumed to be piecewise constant . Estimating the boundary of the partition is the most refined and difficult task in these settings . Thus , any of the many scenarios in which biclustering is relevant can be used to motive our task . An analogous observation holds also for the more general problem of identifying an anomalous cluster ( sub-graph ) in a network , a problem that has been tackled ( for testing purposes only ) by [ 4 ] , the reference therein provide numerous examples of applications . On a high-level , the relationship between the partition and signal recoveries can be thought of the relationship between the estimation consistency and support consistency in a high-dimensional linear regression problems . They can be done by almost identical algorithms but the theoretical results rely on different sets of conditions . The paper is organized as follows . In the rest of this section we formalize the problem settings and the task of partition recovery , and describe the DCART procedure . We further summarize our main findings and discuss related literature . Section 2 contains our main results about one- and two-sided consistency of DCART and its modification . In Section 2.3 we derive a minimax lower bound stemming from the case of one rectangular region of elevated signal against pure background . Illustrative simulations corroborating our findings can be found in Section 3 . The Supplementary Material contains the proofs . Notation We set N = nd , the size of the lattice Ld , n , where we recall that d ≥ 2 is assumed fixed throughout . For any integer m ∈ N∗ , let [ m ] = [ 1 , m ] . Given a rectangular partition Π of Ld , n , let S ( Π ) be the linear subspace of RLd , n consisting of vectors with constant values on each rectangle in Π and let OS ( Π ) ( · ) be the orthogonal projection onto S ( Π ) . For any R ⊂ Ld , n and θ ∈ RLd , n , let θ̄R = |R|−1 ∑ i∈R θi , where | · | is the cardinality of a set . Two rectangles R1 , R2 ∈ Π are said to be adjacent if there exists l ∈ [ d ] such that R1 and R2 share a boundary along el and one is a subset of the other in the hyperplane defined by el , the lth standard basis vector in Rd . See Definition S1 for a rigorous definition . This concept of adjacency is specifically tailored to – and in fact only valid for – dyadic ( and hierarchical , in the sense specified by [ 12 ] ) rectangular partitions , which are most relevant for this paper . For any subsets A , B ⊂ Ld , n , define dist ( A , B ) = mina∈A , b∈B ‖a − b‖ . Throughout this paper , we will use the ` 2-norm as the vector norm . 1.1 Problem setup . We begin by introducing two key parameters for the model specified in ( 1 ) and a well-defined notion of rectangular partition induced by θ∗ . Definition 2 ( Model parameters , induced partitions ) Let θ∗ as in ( 1 ) and { R∗j } j∈ [ m ] be a rectangular partition of Ld , n associated with θ∗ . Consider the graphG∗ = ( E∗ , V ∗ ) , where V ∗ = [ m ] and E∗ = { ( i , j ) : θ̄∗R∗i = θ̄ ∗ R∗j , R∗i and R ∗ j are adjacent } . Let { C∗l } l∈ [ L ] be all connected components of G∗ and define Λ∗ = Λ∗ ( θ∗ ) = { ∪j∈C∗1R ∗ j , . . . , ∪j∈C∗LR ∗ j } as the partition ( not necessarily rectangular ) induced by θ∗ . We say that the union of rectangles ∪j∈C∗sR ∗ j and ∪j∈C∗t R ∗ j , s , t ∈ [ L ] , s 6= t , are adjacent , if and only if there exists ( i , j ) ∈ C∗s × C∗t such that R∗i and R∗j are adjacent . Let κ and ∆ be the minimum jump size and minimal rectangle size , respectively , formally defined as κ = min a∈A , b∈B , A , B∈Λ∗ , θ∗a 6=θ∗b |θ∗a − θ∗b | and ∆ = min j∈ [ m ] |R∗j | . It is important to emphasize the difference between a partition associated with θ∗ , as described in Definition 1 , which may not be unique , and the partition Λ∗ induced by θ∗ of Definition 2 , which is instead unique and thus describes a well-defined functional of θ∗ . The parameters κ and ∆ capture two complementary aspects of the intrinsic difficulty of the problem of estimating Λ∗ ; intuitively , one would expect the partition recovery task to be more difficult when κ and ∆ are small ( and σ is large ) . Below , we will prove rigorously that this intuition is indeed correct . When d = 1 , both parameters , along with σ , have in fact been shown to fully characterize the change point localization task : see , e.g. , [ 36 , 35 ] . The partition recovery task can therefore be formulated as that of constructing an estimator Λ̂ of Λ∗ , the induced partition of θ∗ , such that , as the sample size N grows unbounded and with probability tending to one , |Λ̂| = |Λ∗| and ∆−1dHaus ( Λ̂ , Λ∗ ) = ∆−1 max A∈Λ∗ min B∈Λ̂ |A4B| → 0 , ( 3 ) where A4B is the symmetric difference between A and B . We refer to dHaus ( Λ̂ , Λ∗ ) as the localization error for the partition recovery problem . The dyadic classification and regression trees ( DCART ) estimator . In order to produce a computationally efficient estimator of Λ∗ satisfying the consistency requirements ( 3 ) , we deploy the DCART procedure [ 14 ] , which can be viewed as an approximate solution to the problem in ( 2 ) . Instead of optimizing over all vectors in RLd , n , DCART minimizes the objective function only over vectors associated with a dyadic rectangular partition , which is defined as follows . LetR = ∏ i∈ [ d ] [ ai , bi ] ⊂ Ld , n be a rectangle . A dyadic split of R chooses a coordinate j ∈ [ d ] , l the middle point of [ aj , bj ] , and splits R into R1 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ aj , l ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] and R2 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ l+1 , bj ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] , with [ 0 ] = [ c2 , c1 ] = ∅ , c2 > c1 . Assuming that n is a power of 2 , starting from Ld , n itself , we proceed iteratively as follows . Given the partition { Ru } u∈ [ k ] , one chooses a rectangle Ru and performs a dyadic split on Ru that leads to the largest reduction in the objective function . Any partition constructed through a sequence of such steps is called a dyadic rectangular partition . With a pre-specified λ > 0 , the DCART estimator is θ̃ = OS ( Π̃ ) ( y ) , where Π̃ ∈ arg min Π∈Pdyadic , d , n { 2−1‖y −OS ( Π ) ( y ) ‖2 + λ|Π| } , ( 4 ) where Pdyadic , d , n is the set of all dyadic rectangular partitions of Ld , n . As shown in [ 14 ] and [ 12 ] , the DCART estimator can be obtained via dynamic programming with a computational cost of O ( N ) . Given any solution to ( 4 ) , a natural ( though , as we will see , sub-optimal ) estimator of the induced partition of θ∗ is Π̃ , the partition associated with the resulting DCART estimator θ̃ . Importantly , by the property of DCART , and using the fact that the Gaussian errors have a Lebesgue density , Π̃ is in fact a dyadic-rectangular partition and is unique with probability one , and thus the resulting estimator is well-defined . ( Equivalently , the partition associated with θ̃ and the one induced by θ̃ coincide . )

Summary: In this work the authors consider the problem of learning the partition of a piecewise constant signal supported on a $d$-dimensional lattice $\{1, \ldots, n\}^d$.  They assume access to a noisy signal $y = \theta^\ast + \epsilon$, where $\epsilon \sim \mathcal{N}(0, \sigma^2 I)$ is a Gaussian noise vector and $\theta^\ast$ is the true signal that is assumed to be piecewise constant over a rectangular partition of the latice.  The goal is to recover the underlying partition.  This work focuses on the performance of (variants of) the Dyadic CART (DCART) algorithm for the task of partition recovery.  Prior work [1] has shown that the DCART estimator achieves the optimal error for the task of recovering the true signal $\theta^\ast$.  The authors argue that the partition produced by the vanilla DCART algorithm may contain a large number of rectangles.  However, they show that inside each rectangle of the partition of the vanilla DCART, the signal is indeed almost constant.  The second result of this work proposes simple modifications for DCART so that the corresponding partition is close to the true underlying partition.  In particular they enforce a lower bound on the size of the rectangles of DCART and, at the end, merge cells that are close and have similar value.  They show that the modified DCART algorithm outputs a partition at distance roughly $\sigma^2 k^\ast \log(N) / \kappa^2$, where $N = n^d$ is the size of the grid, $\kappa$ is the minimum signal difference between contiguous cells of the true signal, and $k^\ast$ is the minimum number of dyadic cells required to create a partition consistent with the true signal.  The authors show a lower bound of $\sigma^2 \log(N)/\kappa^2$ for the error. Therefore, their upper and lower bounds differ by a factor of $k^\ast$.  Finally, the authors are able to remove the extraneous factor $k^\ast$ under the additional assumption that the underlying partition is not very dense, i.e., around every rectangle of the true underlying partition there are not many other rectangles.   [1]: Sabyasachi Chatterjee and Subhajit Goswami Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees