Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: The paper proposes a straightforward method for end-to-end learning of active contours, based on predicting a dense field of 2D offsets, and then iteratively evolving the contour based on these offsets. A differentiable rendering formulation by Kato et al is employed to make the process of aligning a contour to a GT mask differentiable. 

The model shows rather compelling results on small datasets, and is very simple, with very strong parallels to active contours, which is a strength. The results improve those of DARNet, which to the best of my knowledge is the main published work in the space other than Curve-GCN. One thing that would be helpful, is  to have an experiment on a large dataset, such as Cityscapes -- right now all the datasets are testing the model in only the small-data regime. Perhaps in a supplement, it would also help to do ablation of how input image / dense deformation resolution affects the result quality -- the input can be subsampled by powers of 2 for the experiment. 

As Amlan Kar helpfully points out, the work heavily overlaps with his approach "Fast Interactive Object Annotation with Curve-GCN", CVPR 2019, which is not cited or compared to. Curve-GCN similarly utilizes differential rendering (only a different variant) to match the GT masks. To me, the main difference wrt Curve-GCN is that explicit dense displacement fields are generated by the net and used directly for the iterative refinement steps, while Curve-GCN leverages implicit feature embeddings and uses GCN layers for their iterative updates. A second main difference is that Curve-GCN supports splines and interactive editing, while the proposed approach does not. Beyond these, there are multiple other differences that the authors point out, but those are more of a technical nature. Unfortunately, without a more direct comparison, it is very difficult to evaluate the design choices in the two approaches, which I feel is necessary for proper understanding of the paper. 

AFTER REBUTTAL: The authors made additions that covered my concerns, so I have switched my recommendation. 

A few more minor clarity / presentation issues. 
-- “The recent learning-based approaches are either non-competitive or proven to be effective in the specific settings of building segmentation". It's not exactly clear what the point is in the context. Which "learning-based approaches"? 
-- Typo 'backpropogation'. 
-- A little better explanation of how a differentiable renderer of Kato works would have been helpful. 
-- Figure 3 is not referenced in the text, takes a little bit of thought why it is relevant (helps explain Fig 1, but maybe better to show it prior to Fig 1). 
-- In Eq 4 it’s not clear what F is.  (I see it is explained in Algorithm box, but that's much later)






review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: This paper investigates an image segmentation technique that learns to evolve an active contour, constraining the segmentation prediction to be a polygon (with a predetermined number of vertices).  The advantage of active contour methods is that some shapes (such as buildings) can naturally be represented as closed polygons, and learning to predict this representation can improve over pixelwise segmentation.

The authors propose to learn an image-level displacement field to evolve the contour, and a neural mesh renderer to render the resulting mask for comparison with the ground truth mask.  The performance compared to prior learning-based active contour methods is impressive.

In section 4.3, there’s a reference to a “gap in performance” between the proposed method and DARNet and a reference to a "low number of vertices," but a comparison between the two methods as the numbers of vertices is varied seems to only be present in Fig. 6 -- it would be interesting to see an explanation of the discrepancy for the lower number of vertices seen in this figure.

Overall, due to the relative simplicity of the approach and impressive performance compared to prior learning-based approaches I recommend to accept.

Post-rebuttal:  I maintain my recommendation.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: EDIT: The rating changed from '1: Reject' to '6: Weak accept' after the rebuttal. See below for my reasoning.

The submission considers two-class image segmentation problems, where a closed-contour image region is to be specified as the 'object'/region of interest, vs. 'no-object'/background. The approach taken here is end-to-end learning with an active-contour type approach. The main loss, in contrast to other active contour approaches, contains a direct difference of the estimated polygon area vs. ground truth polygon area.

The applied method seems conceptually quite simple (as admitted by the authors in Section 5), and the neural rendering approach seems quite neat, but both method presentation (Section 3) and evaluation (Section 4) seem incomplete and leave significant open questions.

One of my main concerns is related to the fact that the displacement field is static and, according to Figure 1 and Algorithm 1, is evaluated only once per image.
If the displacement field J is not conditioned on the current polygon shape (and this does not seem to be the case), then I am wondering why T iterations in the sampling/rendering part are necessary at all. When only considering L_seg, the optimal solution should be found within one iteration, since the displacement field will be able to provide the optimal answer. So maybe these iterations are only necessary when L_B and L_K are incorporated?
In any case, it is unclear why even L_seg is accumulated (using unweighted mean) over all T iterations before being backpropagated. Does this mean that these iterations are not meant to yield shape improvements? Why is ||M^t-M|| not evaluated per iteration, for the purpose of minimization?
It is also not sufficiently clear whether M^t in Equation 4 is a filled polygon mask, or if the mask is just related to the boundary (with a certain width). In absence of explanatory image material, I am assuming the former.
Overall the method description remains weak, since obvious questions/concerns such as the above are not addressed.

The experimental results look good from a quantitative point of view, and indeed, the strongest baselines, e.g. DARNet, are outperformed significantly in many cases.
Section 4 mostly focuses on quantitative evaluation and lots of picture examples, but fails to give insight into particular behaviors, failure cases, etc.
The evaluation procedure is cast a bit into doubt by two things: 1) In Figure 4, the initializations (blue circles) between the DARNet method and the proposed method are very different in size. I am wondering if this then still constitutes a fair comparison, and I have some doubts there. 2) In Figure 6, the proposed method consistently looks much worse than the DARNet baseline (and, in contrast to the baseline, completely fails for 4 vertices), unless the colors were swapped in the description.

Overall, I do not think the submission is in a good enough shape for acceptance.

Minor remarks:
- The values for lambda_1 and lambda_2 seem to come out of thin air, and they also seem quite small. It needs to be mentioned how they were determined.
- Data augmentation by rotation seems to be missing several values (between 270 and 260 degrees) and also not evenly spaced. Is this a typo or on purpose? In the latter case, an explanation is needed, since this seems weird.
- Section 4.3: There is no "Figure 4.2", I assume you mean Figure 6, which otherwise remains unreferenced.
- Section 4.3, Ablation Study: Don't use the word "derivatives" when you're talking about variations.
- Section 4.3, Ablation Study: "even without no auxiliary loss" -> remove "no" or change "without" -> "with"

-------------
Post-rebuttal comments:

I have read the revised version, as well as the other reviews and all authors' comments. The inclusion of an evaluation on a larger-size data set is highly appreciated, and seems to indeed validate the robustness of the method. Typos were fixed, including the switched color descriptions in Figure 7 (which should not have passed initial submission in the first place, if the text had been proofread properly).

Several of the open questions (e.g. "Why is L_seg accumulated before backpropagation?", "Why is the algorithm iterative if the displacement map is computed only once, if not for the other loss terms?", "Choice of values for lambda_1, lambda_2", Initial diameter of initialization") have been somewhat addressed by the authors in the rebuttal comment, though not in great detail.

Based on the quality of the results across data sets, and because I believe that the timely publication of this rather simple method can benefit further research in this area, I have adjusted my score to a 'Weak accept'. That said, I still do not think it is a good manuscript, and my score should be seen as a massive benefit of the doubt toward the authors.

Most importantly, above questions have NOT been adequately addressed in the actual revised text. The authors claim they have "improved the manuscript considerably", but yet I see more reasoning for certain choices described in the comment here than in the actual manuscript. Most of the changes are in Section 2 and the new Section 4.3, but not much relevant to my comments changed in Section 3.

For example, balloon and curvature losses aside, it is still not clear why an iterative approach would be helpful past the first iteration. An ideal displacement map that is not conditioned on the polygon should point, for each pixel, straight to the closest contour pixel. It is clear to me that this may not be what is being learned when multiple iterations are forced, yet it is not addressed why multiple iterations should be beneficial. (I could see why they could be beneficial if the approach was conditioned on the polygon vertices, to avoid vertex collapsing, but it's not.)

A good submission preempts these kinds of questions by addressing them carefully. What seems crystal clear to the authors will not be crystal clear to every reader. The authors should be more careful to include their reasoning in the actual text, which I believe this is essential for proper, easy understanding of the paper.
review_title: Official Blind Review #3

Meta-review: The submission presents a differentiable take on classic active contour methods, which used to be popular in computer vision. The method is sensible and the results are strong. After the revision, all reviewers recommend accepting the paper.
Decision: Accept (Poster)