Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper focuses on perceptual organizatio where the goal is to perceive and group the individual visual element. To address the visual ambiguity in the discrimination process, authors propose to use e figure-ground assignment mechanism to improve deep neural networks  to achieve a robust perceptual organization. Specifically, a Figure-Ground-Aided (FGA) module is presented to learn the configural statistics of the visual scene and leverage it for the reduction of visual ambiguity. An Interactive Enhancement Module (IEM) is designed to leverage configural cues from FGA to enhance the representation learning. Experiments are conducted on four applications.
strengths_and_weaknesses: Strength:

+ The idea of taking the inspiration from the perceptual organization of human vision and studying the figure-Ground cues is interesting. The motivation is also well explained. 

+ Experiments are conducted on different robust perception organization tasks, such as camouflaged object detection, polyp segmentation, and lung infection segmentationm. The results validate that the proposed method can help improve the performance of robust perception organization tasks. 

+ The proposed Figure-Ground Segregation test can help to investigate the performance of models in Figure-Ground assignment.


Weakness:

- The performance gaps between the proposed method and existing methods are marginal in some metrics. The best numbers are not always marked in bold which is missleading, e.g. second row of "Figure 9", second row of "Figure 10".

- The main experimental result tables are titled as Figures, such as Figure7, 8, 9, 10. To me, they are tables rather than fingures. 
questions: Please refer to the weakness part in theStrengths And Weaknesses section.
limitations: Authors discussed the limitations of the method in the main paper and the appendix.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a module for enhancing figure-ground assignment, and which can be used to extend existing encoder-decoder based segmentation models. More specifically, the authors implement mechanisms for measuring two figure-ground cues, namely ‘convexity’ and ‘lower region’. These two cues come from the cognitive vision literature, implemented via morphological operators, and inserted into the encode-decoder CNN architecture. Experimental evaluation of known CNNs augmented with this novel module are then performed on challenging segmentation tasks, e.g., those of medical imaging, indicating an improvement of accuracy in most tasks.
strengths_and_weaknesses: Strengths: 
•	Figure-ground assignment is indeed a fundamental process for computer vision algorithms, which is yet not fully explored. I like the idea of adding FG mechanisms to the current FCN segmentation models.
•	The FG segregation test is an important test, which is indeed popular on the human vision literature. I agree that segmentation models such as DeeplabV3 should be able to replicate human behavior in this test set, with and without additional training from a similar set. 

Weaknesses:
•	The paper is a bit hard to follow, and several sections were needed more than one reading pass. I suggest improving the structure (introduction->method->experiments), and put more focus on the IEM in Fig 3, which is in my view the main figure in this paper. Also, to improve the visualization of the Fig 7, and Fig. 10. 

questions: The IEM is built upon adding more learning units through the spatial self-attention mechanism. I wonder if adding parameters, and/or depth can improve the IoU and S-measure on the FG test set. Can you comment on the number of parameters added by the IEM units to the ResNet50, and speculate about performance of deeper segmentation models?
limitations: It will be good to exemplify few failure cases of your model (e.g., on the FG or medical datasets). Perhaps other FG factors are needed? (e.g., good continuation?).
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a Figure-Ground-Aided (FGA) module to improve the performance of existing CNN models in figure-ground segregation. In particular, the method considers two figure-ground assignment cues, i.e., convexity and lower-region, and derives two supervisory signals automatically from ground-truths for network supervision. An interactive enhancement module is proposed to enhance feature representations. The results look good.
strengths_and_weaknesses: Strengths

This work is on addressing figure-ground segregation of situations with high visual ambiguity. The paper overall is well organized and easy to read in most sections (not in the technical part). Experiments on a synthetic benchmark and four practical applications are surely extensive and the results look good to me.

Weaknesses

While it makes sense to me to formulate perceptual theory into neural networks, the motivation to use the specific two cues (convexity and lower-region) instead of many other cues (size, surroundedness, orientation and contrast, symmetry, parallelism, meaningfulness) as mentioned in Related Work (Line 343) is not quite clear to me. The two cues explored here have also been explored previously, e.g., in [ref1,ref2], thus the novelty seems to be limited. Regarding the performance, even though that the authors argue that "existing methods degrades significantly when deployed in cases with complex visual ambiguity", I found from Tables 7-10 that the improvements are very minor in most datasets. Thus, I am concerning the core motivation of the method and the effectiveness of the technical implementations. Last, the formulations in Section 2 are not well presented. Though I can get the main point, the poor formulation makes it hard for me to understand all the details. My detailed questions are given in the next section.

[ref1] Sundberg, Patrik, et al. "Occlusion boundary detection and figure/ground assignment from optical flow." CVPR 2011

[ref2] Lu, Yao, et al. "Salient object detection using concavity context." ICCV 2011.

questions: - I am particularly concerning  whether the lower-region cue is well connected to the challenge studied in the work. More discussions are demanded. In addition, I am wondering how will the model perform by directly using a edge-aware loss like HED [ref3] instead of $\mathcal{L}_{LR}$.

- I am unsure about the meaning of "stronger supervisory signals" in Line 89. Why is it a weak form by "directly using mask labels as supervision"? This is not convinced to me since many competitors in Tables 7-10 only use ground-truths as supervision, and still obtains very promising performance. 

- The statement "figure-ground assignment ... contributes almost all perception-based tasks" seems an overclaim, or more discussions should be provided to support the argument. 

- So many errors in only seven formulas:
  - Eq.1 and Eq.2 are not well written. What does the symbol $\Leftarrow$ refer to? 
  - It is unclear the meaning fo $GT_{L_2}$ in Eq. 2.
  - How are $K_c$ and $K_L$ determined? Do they set to the same values across the datasets? How will the different values affect the performance?
  - L184 only gives the definition of $Q^{LR}$, however, strictly speaking, $Q^{Con}$ is not defined.
  - It is not clear what the $E$ in Line 187 refers to. 
  - $\mathcal{L}_M$ is not defined. Should it be $\mathcal{L}_D$ in Line 193? 
  - It is also not clear to me how $M$ in Eq. 5 will be used in the network. 
  -  The $\texttt{Lambda}$ in Eq. 3 is very similar to a self-attention operator. Why is $\texttt{Lambda}$ used here? 
  - The symbols $Cue_{LR}$ and $Cue_{Con}$ shown in Fig. 3 are not consistent with symbols in Section 2.
  - $\{D_i\}_{i=1}^5$ in Fig.3 are also not defined.
  - What does the $P_d$ refer to? The symbols is potentially misused since it is defined to denote position embedding in Line 183.

- Typos or grammar errors:
  - "fully connection layer" should be "fully connected layer"
  - "k-th feature" should be "the k-th feature"

[ref3] Xie, Saining, and Zhuowen Tu. "Holistically-nested edge detection." ICCV 2015.


limitations: No limitation is discussed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work proposes a representation learning method to achieve the Perceptual Organization process implicitly. The authors design a new module named as Figure-Ground-Aided (FGA), in which the convexity and lower region cues are leveraged by the Interactive Enhancement Module (IEM). The model takes traditional CNNs as the backbone. The optimization utilizes segmentation labels and binary cross-entropy (BCE) loss. In the optimization, the convexity and lower region cues derived by segmentation labels are also used as the supervision. Experiments based on the synthetic and realistic datasets are designed to verify the effectiveness. Overall, the motivation of this paper is clear and the designed model is technically sound. Moreover, some claims of this paper are not factual.
strengths_and_weaknesses: Strengths:
1. The originality is good. This work is inspired by cognitive researches. The motivation is clear and interesting, while the designed model is novel and technically sound.
2. The paper is organized well and the statements are clear.
3. The significance is enough. This work aims to learn visual representations being discriminative to the confusing objects and backgrounds. This is an essential problem in computer vision.

Weaknesses:
1. Some necessary implementation details are missing:
   (1) The model architecture is not introduced completely. In Figure 3, the intermediate features E^Con_4, ..., E^Con_1 and E^LR_4, ..., E^LR_1 are derived from the feature E_5. However, there is no description for this process.
   (2) The size of structure element K_C is 10 while that of K_L is 5. Why authors select these hyperparameters?
   (3) In the ablation experiments, it is not clear how to implement the variant models (a), (b) and (c) without the IEM module.

2. In the main paper, there is no direct evidence verifying that the proposed model can reducing the visual ambiguousness in realistic datasets. Although a large number of realistic datasets are used in the comparison experiments, only the overall quantitative results are shown in the paper. The improvements of these quantitative results may be unrelated to the reduction of visual ambiguousness. 
  Therefore, the ablation experiments on the realistic datasets are also necessary. Note that the experimental results on the synthetic datasets cannot be considered as a direct evidence for this problem.

3. Some claims of this paper are not factual.
  (1) In line 297, authors claim that “FGA-Net achieves significant performance improvement compared to the second-best method UGTR [67]”. However, UGTR [67] should not be the second-best method according to the results in Figure 7. For example, SINetv2 [10] is better than UGTR [67] for most of the metrics. Moreover, the superiority of the proposed model is not very significant compared with SINetv2 [10].
  (2) In line 325, authors claim that “Quantitative results are listed in Table 8. We can observe that our model consistently outperforms all other contenders across all metrics”. However, the proposed model is inferior than [20] for the metric Sen.

questions: 1. Why authors design the experiments on the synthetic dataset? If the realistic datasets may include some other irrelevant factors for Figure-Ground Segregation, authors should explain this problem explicitly.
2. There are some mistakes in the notations and expressions:
(1) In Eq. (7), the notation L_M has no explanation. 
(2) In line 277, authors state that “We verify the effectiveness of our proposed method on four challenging visual tasks”. However, the following statement only mention three tasks (COD, PS, LIS).
(3) In Figure 7, authors state that “Comparison with 10 SOTA methods ...”. However, only six comparison methods are involved.
(4) In the second line of Figure 10, the best result is not 0.858 but 0.862. The bold style is marked incorrectly. Authors should check this kind of mistake carefully.
(5) In Figure 9, the results of [66] on two different datasets are identical. Please check it.
(6) In line 148, the subscript i has no explanation. 
(7) In line 193, the notations “i" and “I” has no explanation. 
(8) Figures 7, 8, 9 and 10 should be marked as tables.

limitations: The authors give the societal impact in Section 6. However, there is no limitations discussed in this section. The reviewer believes that the requirement of segmentation labels should be a limitation. Specifically, the traditional object detection models only require box labels, which is cheaper than segmentation labels.
review_title: null

Meta-review: metareview: Overall, the reviewers commend the motivation of the approach, the core ideas presented in the paper, and the extensive experiments conducted for four different applications including camouflaged and salient object detection, infection, and polyp segmentation. 

In response to Reviewer fHq8, the authors have mentioned updated results with hyper-parameter tuning, however, they don’t mention which set is used for this purpose. Details on whether the validation set is used or not and how it is chosen are important for the final version. 

In response to Reviewer ghhU, authors have reported new experiments and comparisons, alongside clarifications on motivation and justification for the choices made as part of the approach. 

It appears that the major concerns from reviewers have been addressed in the response and the paper can be accepted after the rebuttal. Authors are suggested to include all the suggested changes in the final version. 
confidence: Certain
Decision: Accept