Lattice partition recovery with dyadic CART

Abstract: OSCAR HERNAN MADRID PADILLA, Yi Yu, Alessandro Rinaldo

Reviews and author responses:

 Thanks for the clarification on point 2!

 Thank you very much for your feedback.

1.  We agree that some more discussions are necessary and we will try to fit some in the revision if space permits.

2.  Thank you very much for this, which is indeed a typo from us.  In Proposition 3, in the definition of $\mathcal{P}_N$, it should have been $\kappa^2 \Delta/\sigma^2 \geq \log(N)/6$.  We will definitely correct this in the revision.

The authors consider the problem of recovering the partition underlying a piecewise constant function over a $d$-dimensional lattice, _i.e._, a signal with support $L_{d,n} = \lbrace 1, \dots, n\rbrace^d$. That is, we wish to recover the structure such that within each rectangular partition of the lattice, the function $\theta^* \in \mathbb{R}^{L_{d,n}}$ is piecewise constant, from noisy measurements ${y_i = \theta_{i}^* + \epsilon_i}$ for $i \in L_{d,n}$ where $\epsilon_i \sim \mathcal{N}(0, \sigma^2)$ i.i.d. A straightforward formulation would be NP-hard, so they the consider the dyadic CART (DCART) algorithm, which simplifies the search space by restricting it to partitions that can be formed by repeated bisections of partitions, starting with a single partition containing all $N = n^d$ points in the lattice.

The authors provide a number of bounds of the form $\sigma^2 k_{\mathrm{dyad}}(\theta^*) \log(N) / \kappa^2$, where $\kappa$ is the minimum jump size (difference between signal in adjacent partitions) of $\theta^*$. First, they show that using DCART, the number of lattice points in recovered partitions where the true signal is not actually piecewise constant scales at most with the above expression ("one-sided consistency"). They then propose a two-step estimator that adds a step merging rectangles in recovered partitions whose measurements are "close" to each other, and show that under certain assumptions, the Hausdorff distance between the recovered (estimated) partition and the actual partition (accounting for the need to merge rectangles to assure definitional uniqueness) scales at most with the above expression. Finally, they provide a simple example to demonstrate an order-matching lower bound.  **Clarity:** The ideas were mostly comprehensible, but I felt that a lot of the definitions would have benefited from more straightforward explanations than were given in Section 1.1. I appreciate that while the ideas are simple, the lattice- and graph-based nature of the problem makes it challenging to keep notation succinct. Nonetheless, I felt that the underlying ideas weren't that esoteric (which is good), but the descriptions made them feel more dense than necessary for a reader without extensive familiarity with this area of research. My suggestion would be that more diagrams like Figures 1 and 2 to explain notions like induced partitions, dyadic splits and the like would be disproportionately helpful.

At the same time, I wanted to acknowledge that there were certainly useful discussions in the paper, for example, the discussions in lines 186–193 and 267–276 are appreciated. Also, the notation itself was consistent and clear throughout, so nothing was ambiguous or notationally confusing, it was just that some of the ideas came across as requiring a lot of abstract setup.

A minor structural suggestion: The definitions of $\kappa$ and $\Delta$ should be separated from the rest of Definition 1, because the notion of induced partitions is a heavy (even if intuitive) concept in its own right.

**Quality:** Theorem 1 looks sound to me.

For Theorem 2, I notice that the assumptions in Assumption 1 have the same scaling as the Theorem 2 result, and I would have appreciated some discussion of this in the main paper (not relegated to supplementary). The assumptions are briefly explained, but the choice of scaling here is not, and one naturally wonders if the result of Theorem 2 follows from whatever is assumed in Assumption 1. I'm sure there is a good explanation for this and it is not a simple matter of, "whatever you assume in Assumption 1 is what comes out in Theorem 2", but I thought this was quite a critical detail and it would be helpful for the authors to address it. On the other hand, the core ideas for the two-step estimation are comfortably explained in lines 231–249 and this is helpful.

For Proposition 3, I would appreciate a discussion of how the example offered relates to Assumption 1. I see that it vacuously satisfies (12) (since no such $A, B$ exist) but the (13) wasn't immediately obvious to me. If I'm not mistaken, Assumption 1 makes the recovery problem easier, so any example for a lower bound that doesn't satisfy the assumption could be too conservative, and hence not relevant to Theorem 2. This is probably a straightforward misunderstanding—I suspect the clarification here will be brief and I look forward to hearing it.

For the experiments, although the experiments are straightforward I think they give some idea of what is going on. However, I felt the discussion of the evaluation was overly brief. What was it about setting 2 that made it more challenging under the $\mathrm{dist}_1$ metric, but not the $\mathrm{dist}_2$ metric? It is good that standard errors were reported, but some of the standard errors are very large—do the error bars between the two comparted methods overlap? The noise level is reported but the "signal level" (I suppose by this I mean $\Delta$) was not clearly indicated—I had to zoom in a lot to see the numbers on the color bars in the second column. I felt that it wasn't clear from the experiments that the theoretical findings of the authors were mirrored in experiments.

I didn't peruse the supplementary material, other than to quickly check Theorem S1 and Assumption S1 and to try and get some idea of how the expressions in Assumption 1 were motivated (I didn't succeed on the last part), so I haven't checked the proofs in detail.

Finally: A quick discussion on (what is known, if anything) about the relationship between the task of estimating $\theta^*$ and partition recovery would be useful. Do better partition recoveries tend to support lower $\lVert \hat\theta - \theta^* \rVert$, or vice versa, or are they so fundamentally different that it is easy to imagine examples where one task is extremely easy and the other is very difficult?

**Originality/Significance.** I am not sufficiently familiar with the surrounding literature to provide a confident assessment of how original this is given the prior art, so the following is mostly a guess. However, the authors were clear about their focus on partition recovery and the case $d \ge 2$, and how this relates to prior work, and I'm satisfied that this is a novel contribution. I think there is a lot to build on here, and I think the direction taken by the authors is a probably a good contribution to our understanding of the recovery of piecewise constant signals. What is much less clear to me is the degree to which the case $d > 3$ is valuable, since all of the examples of applications given are imagery based and in these contexts I'm only aware of up to 3 (spatial) or maybe at most 4 (including color) dimensions. So I couldn't be confident that the authors' extension to general $d$ will be of wide interest.

**Typos:** (for benefit of authors, not part of assessment)
- Line 127: "DACRT"
- Line 129: "of _he_ procedure"
- Line 148: "broader _then_ the ones"
- Line 239: Should $(i,j)$ in the definition of $E$ be $c$ (or vice versa)?
- Line 252: should the second $\lambda_1$ be $\lambda_2$?

_Edit, 02 Sep 2021: Raise rating from 5 to 6, as discussed in comments/responses below._ The paper is almost entirely theoretical in nature and I don't foresee any direct societal impact as a result of this work.

 Many thanks for your response, and I apologise very much for the tardiness in my response. Your remarks are helpful and on account of this I'm very happy to raise my rating to a 6 (was previously 5).

Just a couple more thoughts:

1. I think it would be helpful for the discussion about Assumption 1 and Theorem 2 in the response to be included in the revised paper, but I also acknowledge that there are competing demands on space and different readers will have different confusions, so I'll understand if this doesn't make the cut.

2. The following isn't a rating-critical issue, since it is minor and at a high level the concern I expressed originally was adequately addressed by the response and in the paper itself (with thanks for pointing it out). But I was actually having a bit of trouble deriving $\kappa^2 \Delta \ge \frac16 \sigma^2 \log (N)$ from the definition of $\mathcal{P}_N$, I assume the part where it says that $\kappa \sqrt\Delta/\sigma \ge \sigma^2 \log(N)/6$ (then square both sides yields $\kappa^2 \Delta / \sigma^2 \ge \sigma^4 [\log(N)]^2 / 36$), if a clarification there is possible?

 Thank you very much for your comments.

**NP-Hard estimator**. Our current theory for the NP-Hard estimator also includes the factor $k^*$ in the upper bound. We will add a comment to the paper about leaving for future work to remove $k^*$ from such upper bound.

**Gaps**. We will also add a comment to the paper about leaving the improvement of the gaps for future work.

 

 I would like to thank the authors for their response. About the third point: my question was whether it is possible to remove the $k^\ast$ factor in the error by considering the NP-Hard estimator.  This was meant mostly as an improvement for future versions of this work (in case it is not known/easy to do so).  My main concerns (regarding the gaps and the fact that the proposed algorithm works roughly by merging the cells produced by DCART) mentioned in my review remain and therefore I will maintain my score.

 Thank you very much for your appreciation and constructive comments.  We will endeavour to incorporate all your suggestions in the revision. In the following, we provide point-by-point responses to your comments.

* __On the implication of consistency in Theorem 2__. In fact, we do not claim that Theorem 2 implies consistency in the sense of (3), as Theorem 2 provides an  upper bound on the Hausdorff distance between the true and estimated partitions. However, if it holds that 
$$ \frac{\sigma^2  k_{\mathrm{dyadic}}(\theta^*) \log (N)}{\kappa^2 \Delta }  \rightarrow 0,$$
then we obtain that (3) holds with probability approaching one, as the lattice size grows unbounded.  We will clarify this in the revision.

* __On the complexity analysis__. If the paper is accepted we will add the following sentence: "Notice that the main computational burden is to find the DCART estimator which has a cost of $O(N)$. From the output of DCART, the computation of the quantities in (9)  and (10) can be done in $O(k(\hat{\theta})^2 )$."  In the revision, we shall also add some plots showing the numerical dependence of the upper bounds on the sample sizes.



 Thank you very much for your appreciation and constructive comments.  We will endeavour to incorporate all your suggestions in the revision. In the following, we provide point-by-point responses to your comments.

* __Regarding the relationship between Assumption 1 and Theorem 2__.  Intuitively speaking, the first part of Assumption 1 ensures that if there are two separated regions where the true signal takes on the same value, then the they should be far apart; otherwise, our algorithm would not be able to tell if they should be merged together or keep separated.  The second part of Assumption 1 requires that the signal strength is large enough.  Technically speaking, if the first part of Assumption 1 (eq.(12)) is changed to "whatever'' (denoted by $w$), then the final result of Theorem 2 would be 
$$d_{\mathrm{Haus}}(\widehat{\Lambda}, \Lambda^*) \leq C ((w \wedge \Delta) \vee \sigma^2 \kappa^{-2} k_{\mathrm{dyad}}(\theta^*)\log(N)),$$
where the term $w \wedge \Delta$ is due to the definition of $\Delta$.  This means that the final rate in Theorem 2 is determined jointly by Assumption 1 and an optimal rate.

* __On Proposition 3__.  As you have mentioned, (12) automatically holds for Proposition 3, since in Proposition 3 only two-piece structures are considered.  The condition (13) reads as
	$$\kappa^2 \Delta \geq c k_{\mathrm{dyad}}(\theta^*) \sigma^2 \log(N),$$
	and the distributions considered in Proposition 3 should satisfy (squaring both sides in the definition of $P_N$)
	$$\kappa^2 \Delta \geq (1/6) \sigma^2 \log(N).$$
	Comparing the above two, besides the constants, there is a gap $k_{\mathrm{dyad}}(\theta^*)$.  As a consequence, Assumption 1 indeed makes the problem simpler, by a factor of $k_{\mathrm{dyad}}(\theta^*)$.  We have pointed out this gap in the paper and we leave this as an open problem on whether one can close such a gap.  To be specific, we do not know if one can show the result of Theorem 2 under $\kappa^2 \Delta \geq (1/6) \sigma^2 \log(N)$.  We would also like to point out, this gap is obsertved in other settings, including the seemingly much simpler 1-d lattice partition recovery problems.

* __On the experiments__.  Scenario 2 is challenging under $\text{dist}_1$ because there are two non-square pieces where the true signal is constant and these are being approximated by rectangles.  This approximation creates additional errors, compared to the scenarios where the true pieces are rectangles.  As for the second part of your question, yes, in some cases there is some overlap but overall our method outperforms the competitor. Finally, our definition of $\Delta$ can be difficult to evaluate in particular for Scenario 2.

* __On the relationship between partition recovery and signal estimation__.  We are not aware of clear mathematical statement for general $d$-dimensional lattice.  When $d = 1$, it is shown that fused lasso returns minimax optimal signal estimation but sub-optimal partition recovery; the $\ell_0$ penalized estimator returns both optimal signal estimation and partition recovery.  

* __On the scale of the dimension $d$__.  The methods and theory in our paper hold for fixed but arbitrary $d$.  However, due to the dependence on $d$ of the computational cost, we would recommend more tailored methods for moderately large $d$.



 Thank you very much for your appreciation and constructive comments.  In the following, we provide point-by-point responses to your comments.

* __General comments__.  It is indeed straightforward to merge the outputs of DCART.  In fact, it is interesting to achieve an optimal partition recovery with this simple remedy.  We agree the gap of $k^*$ is not perfect and we will consider closing this gap in our future work.  We would like to point out that it is a challenging task to close the gap $k^*$ even in many, seemingly much simpler, 1-D lattice partition recovery (a.k.a. change point detection) problems.	
	
* __On the difference between partition estimation and recovering the whole signal__.  Firstly, there is a wide array of applications focusing on detecting the regions rather than estimating the background signals, especially in surveillance and environment monitoring. Our work is motivated by all the applications/problems considered in the large literature on biclustering, where the underlying signal is assumed to be piecewise constant. Estimating the boundary of the partition is the most refined and difficult task in these settings. Thus, any of the many scenarios in which biclustering is relevant can be used to motive our task. An analogous observation holds also for the more general problem of identifying an anomalous cluster (sub-graph) in a network, a problem that has been tackled (for testing purposes only) by Arias-Castro, Candes & Durand, A. (2011); as we mention in our paper, in that reference the authors provide numerous examples of applications. We will add more detailed applications and motivations should the paper be accepted.  On a high-level, the relationship between the partition and signal recoveries can be thought of the relationship between the estimation consistency and support consistency in a high-dimensional linear regression problems.  They can be done by almost identical algorithms but the theoretical results rely on different sets of conditions.  

* __On inefficient estimators__.  Our theory holds for replacing DCART with inefficient (NP-hard) estimators, as we discussed at the beginning of Section 2. 


In this work the authors consider the problem of learning the partition of a
piecewise constant signal supported on a $d$-dimensional lattice $\{1, \ldots,
n\}^d$.  They assume access to a noisy signal $y = \theta^\ast + \epsilon$, where
$\epsilon \sim \mathcal{N}(0, \sigma^2 I)$ is a Gaussian noise vector and
$\theta^\ast$ is the true signal that is assumed to be piecewise constant
over a rectangular partition of the latice.  The goal is to recover the
underlying partition.  This work focuses on the performance of (variants of)
the Dyadic CART (DCART) algorithm for the task of partition recovery.

Prior work [1] has shown that the DCART estimator achieves the optimal error
for the task of recovering the true signal $\theta^\ast$.  The authors argue
that the partition produced by the vanilla DCART algorithm may contain a large
number of rectangles.  However, they show that inside each rectangle of the
partition of the vanilla DCART, the signal is indeed almost constant.  The
second result of this work proposes simple modifications for DCART so that the
corresponding partition is close to the true underlying partition.  In
particular they enforce a lower bound on the size of the rectangles of DCART
and, at the end, merge cells that are close and have similar value.  They show
that the modified DCART algorithm outputs a partition at distance roughly
$\sigma^2 k^\ast \log(N) / \kappa^2$, where $N = n^d$ is the size of the grid,
$\kappa$ is the minimum signal difference between contiguous cells of the true
signal, and $k^\ast$ is the minimum number of dyadic cells required to create a
partition consistent with the true signal.  The authors show a lower bound of
$\sigma^2 \log(N)/\kappa^2$ for the error. Therefore, their upper and lower
bounds differ by a factor of $k^\ast$.  Finally, the authors are able to remove
the extraneous factor $k^\ast$ under the additional assumption that the
underlying partition is not very dense, i.e., around every rectangle of the
true underlying partition there are not many other rectangles.


[1]: Sabyasachi Chatterjee and Subhajit Goswami
Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees  There is a lot of recent research studying estimation of structured signals;
the model and the problem proposed in this work fit in this line of research.
 I think it is an interesting observation that DCART tends to "over-partition" signals.  
However, I found the proposed fix of (roughly) merging the cells produced by DCART a bit
straightforward.  Another weak point of this work is the gap of $k^\ast$ in the
upper and lower bounds.  I think it would be interesting to bound the error of
some (possibly) innefficient estimator for this problem in order to have a
better benchmark for the perfomance of DCART.  Overall, the paper is
well-written and the presentation of the results and assumptions clean.  Since
the problem is interesting and a well-known algorithm is studied, I believe that
this work is above the threshold but I would not push for it to be accepted.


Questions/Comments

I believe it would be beneficial for the paper to add some more high-level
discussion on the difference of partition estimation and recovering the whole
signal, applications of partition recovery, etc. 

Is it easier to bound the error of some inefficient estimator (for example the NP-Hard
estimator of Equation (2)) for this problem?  The authors adequately addressed the limitations and potential negative societal impact of their work.

This article investigates the problem of the recovery of constancy regions of a noisy high dimensional piecewise constant signal. The authors proposed to use the DCART estimator to build up a rectangular partition and proved a weak consistency result. Then, they proposed a two steps estimator based on an appropriate merging of the rectangles that DCART output. For this variant (DCARTAM), they proved consistency: the estimator converges to the ground truth partition in the sense of Hausdorff (after an appropriate normalization by the number of rectangles in the partitions). 
Finally, DCART and DCARTAM were compared to the TV-based estimator empirically using two metrics. The first one is the Hausdorff distance between the estimator and the ground truth partition, and the second one is the difference between the cardinality of the estimator and the cardinality of the ground truth partition. Visually, DCARTAM seems to be superior to DCART. Moreover, DCARTAM seems to be superior to the TV-based estimator using the second distance, yet it does not show a systematic superiority if the first distance was used.  Overall, the big picture and the intuitions behind the results were well-explained. Moreover, the empirical results look promising. However, the statements of the results are heavily technical and ambiguous: for example, it is unclear how the result of Theorem 2 implies the consistency of DCARTAM: the upper bound (15) ( even taking into account the inequality (14)) is not enough to prove the consistency as defined by (3).
There are some further remarks/suggestions:
- In practice, the two steps estimator requires the evaluation of (9) and (10) to be implemented, which seem to be computationally expensive. It would be interesting to add a complete analysis of the complexity of this algorithm.
- It would be interesting to illustrate the optimality of the upper bounds by numerical simulations that show the dependency on N. No.

Meta-review: This paper studies the problem of recovering the partition of a piecewise constant signal supported on a multi-dimensional lattice.
Specifically, the true signal is assumed to be piecewise constant over an unknown rectangular partition of the lattice. Given access to noisy measurements, the goal is to recover the underlying partition. This work analyzes the performance of the Dyadic CART algorithm for this task. Overall, modulo some issues with the presentation, the reviewers agreed that this work is slightly above the acceptance threshold.

Decision: accept