Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper investigates the problem of modeling insideness using neural networks. To this end, the authors carefully designed both feedforward and recurrent neural networks, which are, in principle, able to learn the insideness in its global optima. For evaluation, these methods are trained to predict the insideness in synthetically generated Jordan curves and tested under various settings such as generalization to the different configuration of curves or even different types of curves. The experiment results showed that the tested models are able to learn insideness, but it is not generalizable due to the severe overfitting. Authors also demonstrated that injecting step-wise supervision in coloring routine in recurrent networks can help the model to learn generalizable insideness. 

This paper presents an interesting problem of learning to predict insideness using neural networks, and experiments are well-executed. However, I believe that the paper requires more justifications and analyses to convince some claims and observations presented in the paper. More detailed comments are described below.

1. Regarding the usefulness of learning insideness to improve segmentation
The authors motivated the importance of learning insideness in terms of improving segmentation (e.g., instance-wise segmentation). However, I believe that this claim is highly arguable and needs clear evidence to be convincing. Although I appreciate the experiments in the supplementary file showing that some off-the-shelf segmentation models fail to predict insideness, I believe that these two are very different tasks (one is filling the region inside the closed curve and the other is predicting the labels given the object texture and prior knowledge on shapes; please also note that segmentation masks also can be in very complex shapes, where the prior on insideness may not be helpful). It is still weak to support the claim that learning to predict insideness is useful to improve segmentation. 

2. More analyses for experiment results      
In the experiment, the authors concluded that both feedforward and recurrent neural networks are not generalized to predict insideness in fairly different types of curves. However, it is hard to find further insights in the experiments, such as what makes it hard to generalize this fairly simple task. Improving generalization using step-wise supervision in a recurrent neural network is interesting but not surprising since we simply force it to learn the procedure of predicting insideness using additional supervision. 

To summarize, although the problem and some experiment results presented in the paper are interesting, I feel that the paper lacks justifications on the importance of the problem and insights/discussions of the results. 

review_title: Official Blind Review #5

final_score: rating: 6: Weak Accept
review_content: The paper shows that deep-nets can actually learn to solve the problem of "what is inside a curve" by using a sort of progressive filling of the space outside the curve. The paper suceeds in explaining that and in pointing out the limitations if standard learning to address this problem.

However, 
(1) the proposed demonstration is based in ideal (continuous, noiseless) curves. What would happen in actual (discontinuous, noisy) curves?. What implications does this has in the requirements of the network.
(2) I think more connections to classical and current algorithms are required. For instance:
(2.1) The proposed demonstration (and the arguments of Ullman) reminds me of classical watersheed algorithms [see 2.1]. What is the gain of using deep networks with regard to rather old techniques?. Advantages are not clear in the text.
(2.2) What about connections to recent algorithms of automatic fill-in of images of contours based on conditional GANs [see 2.2]. It seems that these recent techniques already solved the "insideness" problem and even learnt how to fill the inside in sensible ways...
Then, what is the gain of the proposed approach?. 

REFERENCES:

[2.1] Fundamenta Informaticae 41 (2001) 187–228
The Watershed Transform:  Definitions, Algorithms and Parallelization Strategies
Jos B.T.M. Roerdink and Arnold Meijster
http://www.cs.rug.nl/~roe/publications/parwshed.pdf

[2.2] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros
ICCV 2017 https://arxiv.org/abs/1703.10593
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This submission introduces a new concept, termed insideness, to study semantic segmentation in deep learning era. The authors raise many interesting questions, such as (1) Does deep neural networks (DNN) understand insideness? (2) What representations do DNNs use to address the long-range relationships of insideness? (3) How do architectural choices affect the learning of these representations? This work adopts two popular networks, dilated DNN and ConvLSTM, to implement solutions for insideness problem in isolation. The results can help future research in semantic segmentation for the models to generalize better. 

I give an initial rating of weak accept because I think (1) This paper is well written and well motivated. (2) The idea is novel, and the proposed "insideness" seems like a valid metric. This work is not like other segmentation publications that just propose a network and start training, but perform some deep analysis about the generalization capability of the existing network architectures. (3) The experiments are solid and thorough. Datasets are built appropriately for demonstration purposes. All the implementation details and results can be found in appendix. (4) The results are interesting and useful. It help other researchers to rethink the boundary problem by using the insideness concept. I think this work will have an impact in semantic segmentation field. 

I have one concern though. The authors mention that people will raise the question of whether these findings can be translated to improvements of segmentation methods for natural images. However, their experiments do not answer this question. Fine-tuning DEXTR and Deeplabv3+ on the synthetic datasets can only show the models' weakness, but can't show your findings will help generalize the model to natural images. Adding an experiment on widely adopted benchmark datasets, such as Cityscapes, VOC or ADE20K, will make the submission much stronger. 






review_title: Official Blind Review #2

Meta-review: This paper investigates a notion of recognizing insideness (i.e., whether a pixel is inside a closed curve/shape in the image) with deep networks. It's an interesting problem, and the authors provide analysis on the limitations of existing architectures (e.g., feedforward and recurrent networks) and present a trick to handle the long-range relationships. While the topic is interesting, the constructed datasets are quite artificial and it's unclear how this study can lead to practically useful results (e.g., improvement in semantic segmentation, etc.). 
Decision: Reject