Paper:
Image segmentation aims at grouping pixels that belong to the same object or region. At the heart of image segmentation lies the problem of determining whether a pixel is inside or outside a region, which we denote as the "insideness" problem. Many Deep Neural Networks (DNNs) variants excel in segmentation benchmarks, but regarding insideness, they have not been well visualized or understood: What representations do DNNs use to address the long-range relationships of insideness? How do architectural choices affect the learning of these representations? In this paper, we take the reductionist approach by analyzing DNNs solving the insideness problem in isolation, ie. determining the inside of closed (Jordan) curves. We demonstrate analytically that state-of-the-art feed-forward and recurrent architectures can implement solutions of the insideness problem for any given curve. Yet, only recurrent networks could learn these general solutions when the training enforced a specific "routine" capable of breaking down the long-range relationships. Our results highlights the need for new training strategies that decompose the learning into appropriate stages, and that lead to the general class of solutions necessary for DNNs to understand insideness.

Review:
This paper investigates the problem of modeling insideness using neural networks. To this end, the authors carefully designed both feedforward and recurrent neural networks, which are, in principle, able to learn the insideness in its global optima. For evaluation, these methods are trained to predict the insideness in synthetically generated Jordan curves and tested under various settings such as generalization to the different configuration of curves or even different types of curves. The experiment results showed that the tested models are able to learn insideness, but it is not generalizable due to the severe overfitting. Authors also demonstrated that injecting step-wise supervision in coloring routine in recurrent networks can help the model to learn generalizable insideness. 
This paper presents an interesting problem of learning to predict insideness using neural networks, and experiments are well-executed. However, I believe that the paper requires more justifications and analyses to convince some claims and observations presented in the paper. More detailed comments are described below.
1. Regarding the usefulness of learning insideness to improve segmentation
The authors motivated the importance of learning insideness in terms of improving segmentation (e.g., instance-wise segmentation). However, I believe that this claim is highly arguable and needs clear evidence to be convincing. Although I appreciate the experiments in the supplementary file showing that some off-the-shelf segmentation models fail to predict insideness, I believe that these two are very different tasks (one is filling the region inside the closed curve and the other is predicting the labels given the object texture and prior knowledge on shapes; please also note that segmentation masks also can be in very complex shapes, where the prior on insideness may not be helpful). It is still weak to support the claim that learning to predict insideness is useful to improve segmentation. 
2. More analyses for experiment results      
In the experiment, the authors concluded that both feedforward and recurrent neural networks are not generalized to predict insideness in fairly different types of curves. However, it is hard to find further insights in the experiments, such as what makes it hard to generalize this fairly simple task. Improving generalization using step-wise supervision in a recurrent neural network is interesting but not surprising since we simply force it to learn the procedure of predicting insideness using additional supervision. 
To summarize, although the problem and some experiment results presented in the paper are interesting, I feel that the paper lacks justifications on the importance of the problem and insights/discussions of the results.