Lattice partition recovery with dyadic CART

1 Introduction . Suppose we observe a noisy realization of a structured , piece-wise constant signal supported over a d-dimensional square lattice ( or grid graph ) Ld , n = { 1 , . . . , n } d. Data that can be modeled in this manner arise in several application areas , including in satellite imagery [ e.g . 30 , 37 ] , computer vision [ e.g . 6 , 38 ] , medical imaging [ e.g . 26 , 23 ] , and neuroscience [ e.g . 16 , 32 ] . Our goal is to estimate the constancy regions of the underlying signal . Specifically , we assume that the data y ∈ RLd , n are such that , for each coordinate i ∈ Ld , n , yi = θ ∗ i + i , ( 1 ) where ( i , i ∈ Ld , n ) are i.i.d . N ( 0 , σ2 ) noise variables and the unknown signal θ∗ ∈ RLd , n is assumed to be piece-wise constant over an unknown rectangular partition of Ld , n . We define a subset R ⊂ Ld , n to be a rectangle if R = ∏d i=1 [ ai , bi ] , where [ a , b ] = { j ∈ Z : a ≤ j ≤ b } , a , b ∈ Z . A rectangular partition of Ld , n , P , is a collection of disjoint rectangles { Rl } ⊂ Ld , n , satisfying ∪R∈PR = Ld , n . To each vector in Ld , n , there corresponds a ( possibly trivial ) rectangular partition . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Definition 1 A rectangular partition associated with a vector θ ∈ RLd , n is a rectangular partition { Rl } l∈ [ 1 , k ] of Ld , n , such that θ takes on constant values over each Rl . For a vector θ ∈ RLd , n , we let k ( θ ) be the smallest positive integer such that there exists a rectangular partition with k ( θ ) elements and associated with θ . In this paper , we are interested in recovering a rectangular partition associated with the signal θ∗ in ( 1 ) . A complication immediately arises when d ≥ 2 : the rectangular partition associated with a given θ is not necessarily unique . This fact is illustrated in Figure 1 , where the left-most plot depicts the lattice supported vector θ , consisting of a rectangle of elevated value ( in grey ) against a background ( in white ) . For such θ , we show three possible rectangular partitions , each of which comprised of five rectangles ( the second , third and fourth plots ) . In fact , the partition recovery problem is well defined , as long as we consider coarser partitions comprised by unions of adjacent rectangles instead of individual rectangles : see Definition 2 below for details . We remark that this issue does not occur in the univariate ( d = 1 ) case , for which the partition recovery task has been thoroughly studied in the change point literature ; see section 1.3 below . Thus , we assume that d ≥ 2 . For the purpose of estimating the rectangular partition associated with θ∗ ( or , more precisely , its unique coarsening as formalized in Definition 2 ) , we resort to the dyadic classification and regression tree ( DCART ) algorithm of [ 14 ] . This is a polynomial-time decision-tree-based algorithm developed for de-noising purposes for signals over lattices , and is a variant of the classification and regression trees ( CART ) [ 7 ] . See Section 1.1 below for a description of DCART . The optimal regression trees ( ORT ) estimator , recently proposed in [ 12 ] , further builds upon DCART and delivers sharp theoretical guarantees for signal estimation while retaining good computational properties – though we should mention that in our experiments we have found DCART to be significantly faster . Both DCART and its more sophisticated version ORT can be seen as approximations to the NP-hard estimator θ1 = arg minθ∈RLd , n { 2−1‖y − θ‖2 + λk ( θ ) } , ( 2 ) where k ( θ ) is given in Definition 1 , ‖ · ‖ is the vector ( or Euclidean ) ` 2-norm and λ > 0 a tuning parameter . DCART modifies the above , impractical optimization problem by restricting only to dyadic rectangular partitions . This leads to significant gains in computational efficiency without sacrificing on the statistical performance . Indeed , decision-tree-based algorithms have been shown to be optimal under various settings for the purpose of signal estimation ; see [ 12 , 20 ] . In this paper , we further demonstrate their effectiveness for the different task of partition recovery . In particular , we show how simple modifications of the DCART ( or ORT ) estimator yield practicable procedures for partition recovery with good theoretical guarantees and derive novel localization rates . Note that , there is a wide array of applications focusing on detecting the regions rather than estimating the background signals , especially in surveillance and environment monitoring . Our work is motivated by all the applications/problems considered in the large literature on biclustering , where the underlying signal is assumed to be piecewise constant . Estimating the boundary of the partition is the most refined and difficult task in these settings . Thus , any of the many scenarios in which biclustering is relevant can be used to motive our task . An analogous observation holds also for the more general problem of identifying an anomalous cluster ( sub-graph ) in a network , a problem that has been tackled ( for testing purposes only ) by [ 4 ] , the reference therein provide numerous examples of applications . On a high-level , the relationship between the partition and signal recoveries can be thought of the relationship between the estimation consistency and support consistency in a high-dimensional linear regression problems . They can be done by almost identical algorithms but the theoretical results rely on different sets of conditions . The paper is organized as follows . In the rest of this section we formalize the problem settings and the task of partition recovery , and describe the DCART procedure . We further summarize our main findings and discuss related literature . Section 2 contains our main results about one- and two-sided consistency of DCART and its modification . In Section 2.3 we derive a minimax lower bound stemming from the case of one rectangular region of elevated signal against pure background . Illustrative simulations corroborating our findings can be found in Section 3 . The Supplementary Material contains the proofs . Notation We set N = nd , the size of the lattice Ld , n , where we recall that d ≥ 2 is assumed fixed throughout . For any integer m ∈ N∗ , let [ m ] = [ 1 , m ] . Given a rectangular partition Π of Ld , n , let S ( Π ) be the linear subspace of RLd , n consisting of vectors with constant values on each rectangle in Π and let OS ( Π ) ( · ) be the orthogonal projection onto S ( Π ) . For any R ⊂ Ld , n and θ ∈ RLd , n , let θ̄R = |R|−1 ∑ i∈R θi , where | · | is the cardinality of a set . Two rectangles R1 , R2 ∈ Π are said to be adjacent if there exists l ∈ [ d ] such that R1 and R2 share a boundary along el and one is a subset of the other in the hyperplane defined by el , the lth standard basis vector in Rd . See Definition S1 for a rigorous definition . This concept of adjacency is specifically tailored to – and in fact only valid for – dyadic ( and hierarchical , in the sense specified by [ 12 ] ) rectangular partitions , which are most relevant for this paper . For any subsets A , B ⊂ Ld , n , define dist ( A , B ) = mina∈A , b∈B ‖a − b‖ . Throughout this paper , we will use the ` 2-norm as the vector norm . 1.1 Problem setup . We begin by introducing two key parameters for the model specified in ( 1 ) and a well-defined notion of rectangular partition induced by θ∗ . Definition 2 ( Model parameters , induced partitions ) Let θ∗ as in ( 1 ) and { R∗j } j∈ [ m ] be a rectangular partition of Ld , n associated with θ∗ . Consider the graphG∗ = ( E∗ , V ∗ ) , where V ∗ = [ m ] and E∗ = { ( i , j ) : θ̄∗R∗i = θ̄ ∗ R∗j , R∗i and R ∗ j are adjacent } . Let { C∗l } l∈ [ L ] be all connected components of G∗ and define Λ∗ = Λ∗ ( θ∗ ) = { ∪j∈C∗1R ∗ j , . . . , ∪j∈C∗LR ∗ j } as the partition ( not necessarily rectangular ) induced by θ∗ . We say that the union of rectangles ∪j∈C∗sR ∗ j and ∪j∈C∗t R ∗ j , s , t ∈ [ L ] , s 6= t , are adjacent , if and only if there exists ( i , j ) ∈ C∗s × C∗t such that R∗i and R∗j are adjacent . Let κ and ∆ be the minimum jump size and minimal rectangle size , respectively , formally defined as κ = min a∈A , b∈B , A , B∈Λ∗ , θ∗a 6=θ∗b |θ∗a − θ∗b | and ∆ = min j∈ [ m ] |R∗j | . It is important to emphasize the difference between a partition associated with θ∗ , as described in Definition 1 , which may not be unique , and the partition Λ∗ induced by θ∗ of Definition 2 , which is instead unique and thus describes a well-defined functional of θ∗ . The parameters κ and ∆ capture two complementary aspects of the intrinsic difficulty of the problem of estimating Λ∗ ; intuitively , one would expect the partition recovery task to be more difficult when κ and ∆ are small ( and σ is large ) . Below , we will prove rigorously that this intuition is indeed correct . When d = 1 , both parameters , along with σ , have in fact been shown to fully characterize the change point localization task : see , e.g. , [ 36 , 35 ] . The partition recovery task can therefore be formulated as that of constructing an estimator Λ̂ of Λ∗ , the induced partition of θ∗ , such that , as the sample size N grows unbounded and with probability tending to one , |Λ̂| = |Λ∗| and ∆−1dHaus ( Λ̂ , Λ∗ ) = ∆−1 max A∈Λ∗ min B∈Λ̂ |A4B| → 0 , ( 3 ) where A4B is the symmetric difference between A and B . We refer to dHaus ( Λ̂ , Λ∗ ) as the localization error for the partition recovery problem . The dyadic classification and regression trees ( DCART ) estimator . In order to produce a computationally efficient estimator of Λ∗ satisfying the consistency requirements ( 3 ) , we deploy the DCART procedure [ 14 ] , which can be viewed as an approximate solution to the problem in ( 2 ) . Instead of optimizing over all vectors in RLd , n , DCART minimizes the objective function only over vectors associated with a dyadic rectangular partition , which is defined as follows . LetR = ∏ i∈ [ d ] [ ai , bi ] ⊂ Ld , n be a rectangle . A dyadic split of R chooses a coordinate j ∈ [ d ] , l the middle point of [ aj , bj ] , and splits R into R1 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ aj , l ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] and R2 = ∏ i∈ [ j−1 ] [ ai , bi ] × [ l+1 , bj ] × ∏ i∈ [ j+1 , d ] [ ai , bi ] , with [ 0 ] = [ c2 , c1 ] = ∅ , c2 > c1 . Assuming that n is a power of 2 , starting from Ld , n itself , we proceed iteratively as follows . Given the partition { Ru } u∈ [ k ] , one chooses a rectangle Ru and performs a dyadic split on Ru that leads to the largest reduction in the objective function . Any partition constructed through a sequence of such steps is called a dyadic rectangular partition . With a pre-specified λ > 0 , the DCART estimator is θ̃ = OS ( Π̃ ) ( y ) , where Π̃ ∈ arg min Π∈Pdyadic , d , n { 2−1‖y −OS ( Π ) ( y ) ‖2 + λ|Π| } , ( 4 ) where Pdyadic , d , n is the set of all dyadic rectangular partitions of Ld , n . As shown in [ 14 ] and [ 12 ] , the DCART estimator can be obtained via dynamic programming with a computational cost of O ( N ) . Given any solution to ( 4 ) , a natural ( though , as we will see , sub-optimal ) estimator of the induced partition of θ∗ is Π̃ , the partition associated with the resulting DCART estimator θ̃ . Importantly , by the property of DCART , and using the fact that the Gaussian errors have a Lebesgue density , Π̃ is in fact a dyadic-rectangular partition and is unique with probability one , and thus the resulting estimator is well-defined . ( Equivalently , the partition associated with θ̃ and the one induced by θ̃ coincide . )

Summary: This paper considers a piecewise constant signal on a $d$-dimensional lattice. The observed signal is corrupted by additive Gaussian noise. The task is to recover the partition of the lattice such that the signal within a partition is constant. The paper proposes an algorithm that builds upon the scalable DCART algorithm.  The contributions of the paper are threefold:  (i)	The authors first analyze the performance of the vanilla DCART algorithm to recover the partitions. They prove the one-sided consistency of the vanilla DCART algorithm. One-sided consistency guarantees that the recovered partitions have large subsets which have constant signal, but a partition of the ground truth signal could potentially be over-partitioned by the DCART algorithm. They demonstrate by an example that it is indeed the case.  (ii)	In order to address the issue of over-partitioning, the authors propose a two-step algorithm where the first step is the vanilla DCART algorithm while restricting to partitions with size above a threshold. In the second step, the partitions which are close and whose signals have a small difference are merged. They prove that this estimator is consistent.  (iii)	They prove that the proposed 2 step estimator is minimax optimal if the number of partitions is constant. Finally, they demonstrate the efficacy of their algorithm by simulations.