Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper proposes the use of RL for instance segmentation. In particular, the paper uses super pixels together with graph partitioning and RL to learn the labels of instances. The model is validated on two datasets, one synthetic and one biomedical imaging dataset and is numerically compared to two approaches ([28] and [47]).
main_review: **Originality:**

In the paper, the main motivation of the use of RL is to enable optimization of discrete objective. However, this use of RL is standard. Moreover, given this paper motivations, the reviewer would expect to see empirical comparisons to alternative methods to learn discrete assignments  (e.g. straight through, DETR-style https://arxiv.org/abs/2005.12872, [8]). Thus, from the methodological point of view the novelty is rather limited. From the application point of view, the instance segmentation of light microscopy images, the paper might have some edge of originality; however, it is hard to assess it given the niche domain. To fully appreciate the originality of the method it would be beneficial to validate the approach on a more standard benchmark. 


**Clarity:**

The paper is generally clearly written. However, the clarity of the methods could be improved. The presentation of this section is a bit convoluted making it hard to grasp the overall picture of the proposed approach. The reviewer would encourage the authors to simplify the presentation of the Methods, especially focusing on the aspects of why particular decisions were made. E.g. why are the authors considering method 1 and 2? Why stateless RL with sequence length of 1? Why not use straight through?

For a non-microscopy expert, it is hard to understand the color codings of Figs 4 and 5. 


**Quality and significance:**

The quality and significance of the results are hard to assess given the niche application domain. It would be beneficial to test the ideas on some more standard benchmarks to instance image segmentation (e.g. COCO). 

Additional questions comments:
- Why are superpixels a good prior for this task? 
- Would it be possible to use other priors/partitioning methods?
- It would be nice to discuss the tradeoffs between added computational costs due to the use of RL and metric gains.
- Would it be possible to report the numbers of a standard instance segmentation model for this task, (e.g. https://arxiv.org/abs/1703.06870 or some followup)?

limitations_and_societal_impact: There is no limitation section/paragraph nor societal impact section/paragraph. Please add those to the conclusions section.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a RL approach for instance segmentation which formulates the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high-level priors on object shape, position and size. 
main_review: This paper presents a novel idea of applying RL methodology to instance segmentation. The stateless actor-critic reinforcement learning theme enables non-differential rewards which could incorporate prior knowledge to guide the segmentation, and relaxes the heavy requirement of pixel-wise labeling. The idea is unique and quite interesting.
The related work looks sufficient to me. The experiments are well designed with convincing results.
The writing is overall clear, but there are a few parts which could be improved:
1. For Section3, it's not clear to me the essential difference between Method 1 & 2. It'd be nice to include a graph as Figure 1 to demonstrate Method 2, instead of only including it in Appendix.
2. There is a lot of valuable information in Appendix. It'd be nice to figure out a way to shorten some parts of Method section (such as Section 3.2 and experiments) and include some content from Appendix here.
3. For the experiments, some failure cases and analysis are necessary. It'd be good to add a section of ablation studies. 

limitations_and_societal_impact: N/A
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: In this paper, the authors proposed a reinforcement learning (RL) framework for instance segmentation with superpixels. The proposed framework is based on a stateless actor-critic setup and is demonstrated to be effective on the instance segmentation tasks for one toy dataset and one microscopy dataset.
main_review: +: strength
-: weakness 

1)Originality: 

+: This paper proposes to solve the instance segmentation tasks with weak supervision (superpixels) using the RL based methods. Different from the typical weakly supervised image instance segmentation methods, the RL-based one is a new and interesting application.

-: The authors only talked about the related work on RL however ignore the related work on instance segmentation (fully supervised and weakly supervised). 

2) Quality:

+: The proposed modules are proved to be effective via the ablation studies on the microscopy dataset. In addition, the authors also provide sufficient analysis on the selection of the hyper-parameters, such as the number of feature channels, random seed, etc.

-: There lack extensive experiments on public instance segmentation benchmark datasets, such as MS COCO [a]. In addition, there are also many instance segmentation tasks in microscopy image analysis, such as nuclei instance segmentation in histopathology images [b], cell instance segmentation in electron microscopy images [1].

[a] https://cocodataset.org/#home
[b] A Multi-Organ Nucleus Segmentation Challenge, in IEEE TMI
[1]: the reference #1 in the manuscript.

3) Clarity:

+: The methods and experiment sections are clearly presented.

-: This paper aims at solving the weakly supervised instance segmentation via RL architecture, however, there lack illustrations and discussion on any related instance segmentation works.

4) Significance:

-: Although the authors have presented many experimental results, there still lack experimental results on representative benchmark instance segmentation datasets on general and medical images, which makes me feel that the overall paper lacks significance. 

limitations_and_societal_impact: 1) The major concern of this work is the lack of experimental results on public benchmark datasets. For the first toy dataset, the background content seems to be fixed (black content and white ‘noise’). For the second microscopy dataset, the morphological structure of the instances is always special: all the instances overlapping together to form a circle-like fruitfly embryo. To this end, the overall experiments on the two datasets are not sufficient to demonstrate the proposed method can be effective on other general datasets, such as the MS COCO. This made the overall impact of the proposed method questionable.  

2) It seems that there are only 10 testing images in the real microscopy dataset, which is a small number. Therefore, a statistical significance test (based on p-value) is suggested for the results on this dataset.

3) Computational complexity analysis is also suggested on the proposed framework.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper presents a strategy for object instance segmentation from images using training data that has no manual annotations. Instead of manual annotations, as is required for many state-of-the-art methods, this paper proposes to use high-level priors represented as rules that can be evaluated in the produced segmentation. These rules define what an acceptable segmentation is, and are used to create a reward signal for a reinforcement learning agent. The framework formulates an environment, an agent, an action space and a reward system in a stateless system to guide segmentation. All the components of the system are well designed and well thought, and this approach may have wide applications in bioimage segmentation problems.
main_review: * The stateless actor-critic model has connections with the dynamics of Generative Adversarial Networks, as the authors note. The fact that there is no state also begs the question of what are the connections with the REINFORCE rule for training models when the loss function is non-differentiable, and how that approach could simplify or enhance the proposed formulation.
* The experimental design is interesting and overall complete. However, the performance metrics used for evaluation are not in line with research in the instance segmentation field, where precision and recall are used in several different ways. For example, computing precision or F1 score at specific intersection over union thresholds would be more informative to understand how many objects / regions are correctly segmented and to what level of overlap with ground truth.
* There are a few datasets studied in the instance segmentation literature that could have been used as benchmarks in this evaluation. At least one dataset used for nucleus segmentation of cells, for instance, would add a lot of value to the paper for two reasons: first, it connects with previous literature in the topic, second, it allows to assess how far a model trained without ground truth data would go for segmenting these objects. It is not expected that the proposed model should be better than fully supervised approaches, but being the first in the unsupervised regime would help understand how far we are from a competitive model while not requiring ground truth data.
* The paper is overall well written and clear.
limitations_and_societal_impact: No comments about limitations. Positive societal impacts expected, as less annotations would be required from experts to obtain accurate models.
review_title: null

Meta-review: This paper proposes to use stateless RL for instance segmentation on microscopy datasets. Initially it received mixed reviews, upon the rebuttal and further discussion, reviewers are more aligned that the approach is novel, however they agreed that the benchmark dataset that the authors used were probably too small. The authors provided an update but that was after the discussion deadline so I don't think that can be considered for this round. Hence AC recommends rejection but the authors should consider adding experiments on public benchmark datasets and submit to the next available venue.
Decision: Reject