Exploring Figure-Ground Assignment Mechanism in Perceptual Organization
Authors: Wei Zhai
Yang Cao
Jing Zhang
Zheng-Jun Zha
Venue: NeurIPS 2022 Accept (2022)
Keywords: Figure-Ground Assignment
Figure-Ground Segregation
Cognitive Inspiration

Abstract: Perceptual organization is a challenging visual task that aims to perceive and group the individual visual element so that it is easy to understand the meaning of the scene as a whole. Most recent methods building upon advanced Convolutional Neural Network (CNN) come from learning discriminative representation and modeling context hierarchically. However, when the visual appearance difference between foreground and background is obscure, the performance of existing methods degrades significantly due to the visual ambiguity in the discrimination process. In this paper, we argue that the figure-ground assignment mechanism, which conforms to human vision cognitive theory, can be explored to empower CNN to achieve a robust perceptual organization despite visual ambiguity. Specifically, we present a novel Figure-Ground-Aided (FGA) module to learn the configural statistics of the visual scene and leverage it for the reduction of visual ambiguity. Particularly, we demonstrate the benefit of using stronger supervisory signals by teaching (FGA) module to perceive configural cues, \ie, convexity and lower region, that human deem important for the perceptual organization. Furthermore, an Interactive Enhancement Module (IEM) is devised to leverage such configural priors to assist representation learning, thereby achieving robust perception organization with complex visual ambiguities. In addition, a well-founded visual segregation test is designed to validate the capability of the proposed FGA mechanism explicitly. Comprehensive evaluation results demonstrate our proposed FGA mechanism can effectively enhance the capability of perception organization on various baseline models. Nevertheless, the model augmented via our proposed FGA mechanism also outperforms state-of-the-art approaches on four challenging real-world applications.

Exploring Figure-Ground Assignment Mechanism in Perceptual Organization
Wei Zhai!
Yang C.o Jing Zhang²
Zheng-Jun Zha'
'University of Science and Technology of China The University of Sydney 'Institute of Artificial Intelligence, Hefei Comprehensive National Science Center {wzhai 056, forrest, zhazj} ( wustc.edu.ccmm jing. zhangl @sydney. edu au
Abstract
Perceptual organization is challenging visual task that aims to perceive and group the individual visual element so that it easy to understand the meaning of the scene whole. Most recent methods building upon advanced Convolutional Neural Network (CNN) come from learning discriminative representation and modeling context hierarchically. However, when the visual appearance difference between foreground and background is obscure, the performance of existing meth- ods degrades significantly due to the visual ambiguity in the discrimination process. In this paper, we argue that the figure-ground assignment mechanism, which con- forms to human vision cognitive theory, be explored empower CNN achieve a robust perceptual organization despite visual ambiguity. Specifically, we present novel Higure-Grrund-AAee statistics of the visual scene and leverage it for the reduction of visual ambiguity. Particularly, demonstrate the benefit of using stronger supervisory signals by teaching (FGA) module to perceive configural cues, i.e., convexity and lower re- gion, that human deem important for the perceptual organization. Furthermore, an Interactive Enhancement Module (IEM) is devised to leverage such configu- ral priors assist representation learning, thereby achieving robust perception organization with complex visual ambiguities. In addition, well-founded visual segregation test is designed to validate the capability the proposed FGA mecha- nism explicitly. Comprehensive evaluation results demonstrate our proposed FGA mechanism can effectively enhance the capability perception organization on various baseline models. Nevertheless, the model augmented via our proposed FGA mechanism also outperforms state-of-the-art approaches on four challenging real-world applications.
(FGA) module to learn the configural
Introduction
Perceptual organization, which is a vital visual task, refers to the processes by which the disjoint bits of visual information are structured into the larger coherent units that we eventually experience as environmental objects. Thanks to the developments of discriminative representation learning and hierarchical context modeling with convolutional neural networks [17, 33, 48, 77], the past few years have witnessed tremendous progress in perceptual organization. Recent advanced methods are usually based on the framework fully convolutional network (FCN) [5. 33, 50, 76], which learn discriminative feature representation and hierarchically model the local context by supervision derived from human- -iven labels.
However, the performance of existing methods degrades significantly when deployed in some challenging tasks with complex visual ambiguity, such camouflaged object detection [11, 27],
"Conresooning author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

medical image segmentation 50], and visual industrial detection [74]. Since the visual appearance differences between the foreground and background are obscure, it is difficult perceive the correlation between individual visual elements and determine the boundaries. The visual ambiguities impede CNN's is represent learning and contextual modeling, leading to inaccurate and incomplete perceptual organization.
To address this issue, more perceptual knowledge is required to be incorpo- rated into visual organization. In the (A)
Percelving
Bleroorath
Grouping Purs leatler
Vinal Elaantts
1920s, the Gestalt psychologists iden- tified Grouping and Higure-Ground as two important principles underly-
ing the process of perceptual organiza- (B)
Parts
tion [24, 25, 53, 66]. Grouping prin- ciple refers to bring together individ- ual visual elements that produce stim- uli to form a holistic perception. has attracted more attention because it is intuitively similar to the contex- tual modeling ability of CNN. Differ- ent from Grouping, Figure-Gree Ground as- signment refers to the perception that assigns a boundary separating two re- gions to of them [52]. By intro- ducing the configural statistics of the natural world in which the visual sys- tem evolved [3]. the Figure-Ground assignment principle is thought to be important in reducing the visual ambiguousness of scene. In addition, Cognitive science studies [15, 47, 81] have found that neural activity associated with the Figure- Grouun assignment mechanism in the V2 cortex of human vision occurs as early as 10-25 ms after the generation of visual stimuli, providing strong support for the role of local bottom-up processing. Another study [43] on the "mean- ingfulness principle" also showed that assigned figures tend to be associated with neighborhoods with familiar shapes, pointing to the integration of knowledge from the top-down.
Perceiving
Grouping
Vioad Elecanti
Parts tigether
(C)
Figure-Ground
Assignment
Figure I: Main idea. (A) The process of perceptual organi- zation first requires perceiving discriminative visual elements, then segregation is obtained by grouping parts together. (B) This process obscured in scenarios with ambiguity. (C) In this paper, we explore Pigure-Ground Assignment to facili- tate reducing visual ambiguities. Best viewed color.
Inspired by these studies, as shown in Fig. I, we argue that exploring figure-ground assignment mechanism can empower CNN the ability of perceptual organization despite visual ambiguity, and consequently presents novel Figure-Ground- Ground-Aided (FGA) module that learns the configural statistics of visual scene, and leverages it for representation learning. Specifically, we firstly investigate the assignment mechanism in human psychophysics and find
configural cues related to the Figure-Groun that the figural region usually takes on the shape instructed by the separating boundary and appears closer to the viewer, while the ground region is seen as extending behind the figure [43]. Typically. the regions on either side of the boundary where the scene depth changes abruptly and is beneficial for analyzing the hierarchical relationship between neighboring regions the image, facilitating hierarchical contextual modeling. And lower region cues [16, 62] usually corresponds to the region of the scene where occlusion has occurred and is beneficial for analyzing the occlusion relationship between various neighborhoods in an image and determining the shape attribution of foreground objects and background regions.
as shown in Fig. 2 (a), Convexity cue [21, 34, 39, 44] corresponds
After that, instead of using weak form of directly using mask labels as supervision. we refer to the method in [32] of teaching the FGA module to perceive the configural cues that human deem important for the perceptual organization by using human-derived labels as stronger supervisory signals. Notably, the Ground-TTre (GT) segmentation masks imply the prior knowledge of the annotators' understanding of natural scenes. Therefore, we easily generate the labels of the two configural cues from the Ground- Truth (GT) label without the intervention of additional information and exploit them as supervisions for the FGA module, facilitating the modeling of the configural statistics of natural scenes. Furthermore, Interactive Enhancement Module (IEM) is presented to progressively enhance the discriminative features for boundary assignment via a local/global interactive strategy. Specifically, collaborative local interaction process is first introduced by swapping queries to align locally between contextual co-occurring configural features. Then, a global

---

interaction process is introduced to establish global spatial correlations of local configural features, resulting in complete object boundaries. Moreover, a Lambda strategy is utilized improve the computational efficiency and performance of the original self-attention module.
To systematically investigate the perfor- mance of our proposed method for the Ggoure assignment, we design a synthetic computer vision task inspired by an important experiment in cognitive science Figure-Ground Segregation [26, 46, 49, 55]. Furthermore, the compre- hensive experiments demonstrate that our proposed FGA module can facilitate the CNN to learn more efficiently in the reduc- tion of visual ambiguities with low data requirements. Finally, we also validate the performance of our proposed mecha- (b) nism in four challenging visual applica- tions, including camouflaged object detec- tion [11, 71], polyp detection [12], and lung anomaly detection [13]. The results demonstrate the superiority of our method over SOTA methods. The contributions are summarized follows:
Cannedyy
F
G
(a)
FP3
B4
#3
#4
LowAr Region
F
Convexity Figure 2: Figure-Ground Cues. (a) The Figure (F)- Ground (G) assignment cues used in this paper. We im- plement the two cues of Convexity and Lower Region as two 2-channel labels, which are directly calculated from GT labels without the intervention of additional information. We represent the concave region and the convex region as different labels to reflect their struc- tural differences for the convexity cues. We represent intra-bounday and extra-boundary regions differently for lower region cues to reflect the differences between the two sides of the boundary. (b) The example of cues. "Convexity" represents the convexity cue. "LR" repre- sents the lower region cue. Best viewed in color.
GT
LR
(I) This paper explores the Figure-Ground Ground assignment mechanism from human vision cognitive theory to empower CNN to learn configural statistics to reduce visual ambi- guities, thereby achieving robust percep- tual organization. (2) This paper presents a novel Figure-Ground- Ground- -iioudd-Aiddd module to
integrate lggure-rouund-idee cues in hi-
erarchical manner. An Interactive Enhance-
ment Module is devised to progressively enhance the discriminative features for Figure-Ground assignment via a local/global interactive strategy. (3) This paper introduces the Figure-Ground Segregation test, synthetic visual perception challenge systematically investigate the perfor- mance of models in Grgure-Ground assignment. Experimental results demonstrate our proposed FGA module consistently improves the Tigure-Ground assignment performance of several representative networks on datasets of different difficulty levels. (4) Extensive experiments are also performed on four challenging robust object segmentation applications, showing that the model constructed via our FGA module outperforms SOTAs.
2 Methodology
Architecture. In this paper, we propose a novel Figure-Ground- Aided (FGA) module that uses con- figuration cues from the Figure-Grous Ground assignment process to enhance the perception of foreground- background relationship. thereby achieving a robust perceptual organization result. It can be easily incorporated into existing mncodrr-decoder models and improve their performance. Fig. shows the overall architecture of the proposed Figure- Saetue--Gronn--iiieee module. Our proposed FGA module composed of two cue-aided branches and an Interactive Enhancement Module (IEM). Here, we use ResNet-50 [17] as the backbone. Specifically, we remove the fully connected layer and retain all convolutional bocks. Given an input image of shape H x W, this backbone will generate five scales of features with spatial resolution gradually decreasing by stride 2. We denote these features W where C₂ is the number as E = (E's A = 1, 2, 3,4,4 4,5.. The size the k-th feature is [Ck, zF 2E of channels of the k-th feature. We utilize the feature from Ex k = 1, 2, 4} as skip-connected feature for FGA module and the decoder. D = (Dx k = 2, 3,23,445 4, 5) are the feature maps of the decoder part. The intermediate features F.Con and F. are derived from the feature Es.. The calculation process here is the same in the Decoder part.
3

---

Interactive Enhancement Module
D
D
Loss
Decoder
Figure- Ground Alded
Lambda
Lambda
E
Encoder
Cart
Imput
Supervisio
C Concatenation
Top-Down informationn Flow
Side Output
Skip Connection
Bottom-Up Information Flow
Figure 3: The architecture of the proposed Figure- Ground Aided module (FGA). Figure-Ground Aided module, which can be easily incorporated into mncoder-decodee models, is composed of two cue-aided branches and an Interactive Enhancement Module (IEM). The IEM is proposed integrate the features at different scales of branches separately to enhance the feature of the decoder branch. We leverage the convexity and lower region cues, which can be calculated directly from the GT label and do not require any additional information, as supervisions for perceiving the contextual information of object boundary, respectively. We first allow the network learn the statistical features related to the configuration in top-down manner. Then we use the learned features to guide the bottom-up features from the encoder to integrate bottom-up and top-down information. Interactive Enhancement Module (IEM), which consists of two parts: Collaborative Local Interaction (CLI) and Global Interaction (GI). IEM has three input features (F,Cn, F,LRR and E), two of which come from the intermediate feature learned by the and ELR). "Lambda" indicates the use of Lambda strategy [1] to improve the computation efficiency of the CLI and GI interaction.
assignment cues aided branches (E
Figure-Ground Assignment Cues. The critical process of the perceptual organization known as figure- grore- assignment [51], involves giving one of the two adjacent regions a boundary. The figure-Grre Griuee assignment is commonly thought to follow region segmentation, and it is an essential step forming perception of surfaces, shapes, and objects [63, 64). The human visual mechanism points out that when humans observe images, they will use some configural cues to distinguish between foreground and background, including Convexity [21, 34, 39, 44] and Lower region [16, 62]. Accordingly, we propose to introduce the convexity and lower region cues, as shown in Fig. 2 (b). To generate the convexity cue, we used morphological opening and closing operations [4, 45J. can be formed as: Cuec Cat Cat(Cue Cuec2), "Cat" is the concatenation operation.
Cuecı GT (GT Kc), Cueça = (GT, Kc) GT,
(1)
where GT' is the ground truth segmentation mask. 4 (.) is the opening operation. (.).) is the closing operation. Ko is the structure element whose size is 10. We get the convexity Cue by concatenating Cueci and Cuec2, whose shape [2, H. W). H and W are the height and width of the mask. To generate the lower region cue, we used morphological erosion and expansion operations. It can be expressed as: Cuez Cat (CueLl, Cue
CueL1 GT o Kt). Cuer2 = o K2) GT,
(3)
(4)
where (.) is an erosion operation. (-) the expansion operation. K2 is the structure element whose size is 5. We get lower region by concatenating Cueri and Cue
Interactive Enhancement Module. To effectively utilize the important configural cues (convexity and lower region) provided by the Figure-Ground assignment mechanism, a novel Interactive En- hancement Module (IEM) devised. The IEM has three input features, two of which come from the intermediate feature learned by the Figure- Ground assignment cuc-aided branches. We define them

---

as E and E respectively. Moreover, the remaining branch is the skip-com connnetnnnctonn feature E from the encoder. We further improve the representation ability the encoder feature by establishing the interactions among the features of the skip-connection features and aided branches feature. As shown in Fig. 3, the interactions in the IFM consist of collaborative local interaction and global interaction. 1) Collaborative Local (CLJ) interaction: the local interaction of the two Figure-Grrund cues integrates the local structural context features. 2) Global interaction (GI): the global interaction between base features and local contextual features aims to enhance foreground features and suppress the background with structural context features.
The detailed structure of the Interactive Enhancement Module is illustrated in Fig. 3. We use spatial self-attention mechanism that considers both content and location interactions to achieve local or global interaction of features. This paper uses an efficient Lambda layer [1], which captures interactions of feature elements by transforming available contexts into linear functions, termed lambdas, and applying these linear functions to each input separately. For the collaborative local interaction part, can be expressed as follows:
Y Lambda (FLR PLR g0l + Lambda (ECoon ppon pC
(5)
where Y is the output of CLL. P denotes the relative position embeddings, QLR is the query, which is computed by: QIR WiR. Q is calculated in the same way. W is the learnable weights. Lambda()) is defined as:
LlamddaEES pote)
WF (E + (pCw) ((E
(6)
where the range of relative position in CLII is set to 5. Cue E LRR, Con For the global interaction part, can be expressed as follows:
M LaandaaE",,P
where E = CY, E and M correspond to the feature maps shown on the right side of the IEM in Fig. 3. The Lambda(.) function is defined as follows:
Lambda pG) (Softmax(F9WR)) (E-W9) + (PO)((F995)
where the range of relative position in GI is set to the whole feature map size.
Loss Function. In this section, introduce the loss function used train FGt-Net. We adopt deep supervision strategy for each sub-side output prediction map from the decoder and calculate the binary cross-entropy (BCH) [40] loss. The total loss LD for the decoder of the proposed network
could be expressed as follows: Lo = =' c(f) where (6)) represents the loss of the i-th sub-side output prediction. For the other two aided branches in FGA module,
also use BCE loss obtain the losses LCon and LLR. Unlike the decoder, we do not use a deep supervision strategy. The final loss for the entire network is defined as follows: Lioua aLp + bLcon + CLLR. where b, and are the hyper-parameters to balance different loss terms. In this paper, we set a = 1, b = 0.9, and 0.9, respectively. Note that all experiments use the same hyper-parameters.
3 Experiments
3.1 Figure-Ground Segregation Test
To investigate the validity of our proposed Figure- Ground Aided module (FGA), we design and estab- lish a set of tasks to evaluate the Prgure-Ground as- signment ability of deep convolutional neural net- works, inspired by the Figure-Ground Ground Segregation test in cognitive science experiments [26, 46, 49, 55] as shown in Fig. (a). This task is described in detail as shown in Fig. 4 (b), by sampling the content of a randomly given texture image and filling the fore- ground and background of the given image using that texture content to generate the desired sample finally.
(a)
Figure 4: Figure-Ground Segregation Test. (a) Exemplars from the Figure-Ground Segre- gation experiment in cognitive research. (b) Exemplars from the FGS test.
Here we use synthetic dataset to test deep convolutional neural network to remove distracting
5

---

Easy
Daty
cas 41
Normal
Harmal
GDeng-aara
4000
8.18
S-measure
% training data
Image
Baseline
Edge
FGA
GT
(a)
(b)
(c)
Figure 5: FGA module has better capability of Figure-Ground assignment. (a) FGA module consistently improves the aigure-Ground assignment performance of several representative networks on all three datasets. Different dash boxes represent different levels. The x-axis and the y-axis are the S-measure and IoU scores, respectively. UNet+ DNet==Edge/Deepl.abye with auxiliary edge supervision. DNNt==GGADDepLaby3-HH
indicates UNet/Deepl.abb means UNet/eeepLabv3 with FGA module. (b) FGA-Net is more data-efficient. The x-axis and the y-axis represent different proportions of the normal dataset and the IoU score, respectively. (c) Visual comparisons on the Normal level dataset. The UNet [50] is used as our baseline model.
factors. Standard computer vision datasets make it difficult to pinpoint the relative contributions of different visual strategies since the performance of architecture may be affected by several factors, including dataset biases, model syper-parameters, and the number of samples [22, 31].
Test Design. We use segmentation labels from the Pascal VOC [8] dataset and the rich texture dataset (DTD [7]) to synthesize our dataset. Referring to the cognitive science experiments which exclude irrelevant factors from the test conditions, such as excluding other irrelevant factors for Figure- Ground Segregation, the influence of contextual information brought by semantic labels since utilizing semantic labels of different objects helps to learn discriminative feature representation; 2) excluding the influence of spatial contexts provided by multiple instances belonging to the same class since they contribute to learning robust (e.g., iavee-invariant segmentation cues due to significant appearance differences between foreground and background since their distinct textures [72. 73], colors, and illumination help to learn cheap features distinguish them. Guided by the above principles, the whole process of sample generation can be divided into the following steps: 1) select a random image in the Pascal dataset and a figure and the remaining regions as the ground; 2) given a collection of texture images, randomly select texture image from it; 3) two random transformations (such rotation and scaling) are performed on the texture image independently; and 4) fill the figure region and the ground region with the two transformed textures respectively, resulting in a synthetic sample. In addition, for more comprehensive evaluation, three datasets of different difficulty levels (Easy, Normal, and Hard) are established by varying the transformations, the division of the set of texture images, and the size of figure regions. Each dataset contains 2,500 unique images with 224 x 224 resolution, split into training (2,000) and test (500) sets.
also need to exclude the influence of several factors, i.e., 1) excluding
occlusion-awarc feature representation; and 3) excluding
of the object instances as
Experimental Conditions. In this paper, we conduct experiment on two representative architectures: U-Net [50] and Deeplabv3 [6]. U-Net is one of the most popular architectures in the field of segmentation, which has typical encoder-decoder architecture, and the shallow features are directly fed into the decoder through skip connections. Deeplabv3 is a highly competitive network for semantic segmentation. Through atrous spatial pyramid pooling, it can expand the receptive field while ensuring detailed modeling. Each model trained using the Adam optimizer with a batch size of 16 and a learning rate of le for the Figure-Ground Ground Segregation test. To isolate the influence of pre- pre-trained weights, all weights are randomly initialized. We train each model for 20,000 iterations. For the evaluation of the results, we focus on the differences between the output and the ground truth in terms of structure and filling completeness, so we choose S-measure [9], and ToU [33] as evaluation metrics.

---

LR
Covx IEM
S.
IoU
Baseline
.715
.580
(a)
.771
.612
(b)
.761
.603
(c)
/
/
.793
.635
FGA
.816
.659
Image Baseline (a)
(b)
(c)
FGA
GT
Figure 6: Ablation study of the proposed FGA-Net on the normal dataset. "Covx" ("LR")
denotes convexity (lower region). The UNet [50] is used as our baseline model.
Result and Discussion. Through experiments on the Figure-Grou Ground segregation test, we can get the following four conclusions. P First, with different difficulty levels, the proposed FGA can consistently improve the performance of the base model (in Fig. (a)). It shows that our method has a strong capability of the Figure- Ground assignment by introducing human cog- nitive cues. Second, as shown in Fig. 5 (a). proposed FGA-Net has a better performance compared to the straightforward strategy of di-
Table 1: Ablation study of IEM on the nor-
mal dataset. "Local" means use local interaction. "Collaborative" means use collaborative strategey. "Global" means use global interaction. "Lambda" indicates the use of Lambda [1] strategy.
Local Collaborative Global Lambda
S.
IoU
801 .640
805 .645
.797 .638
.810
.652
.866 .659
rectly adding edge supervision. Its segmentation results are more complete in terms of both structure and filling of foreground regions as shown in Fig. 5 (c). Using the edge supervision can only provide local perception at the foreground boundary, while more structural priors and perception of the foreground region are required for the Figure-Ground assignment. Third, shown Fig. (b), can be seen that as the number of training samples decreases, our model still outperforms the contenders. Our FGA trained on only 1% of the dataset comparable to Baseline+Edge trained on 10% of the dataset, suggesting that FGA is more data- efficient. The supervisions from the Figure- Ground cues help the network learn an inductive bias [68] that the network focuses on the foreground and background and their differences at the boundaries. thereby reducing the amount of data required. A Fourth, as shown in Fig. 6, the LR cue provides a more complete edge for segmentation results, the Convexity cue enhances the perception of shape detail, and the IEM enhances the consistency within the foreground. In addition, we find that using two branches to learn both priors separately more effective than using a single branch to learn both priors. This phenomenon explains to some extent the certain orthogonality between the two cues, which depend on different features. And we also conduct an ablation study on the IEM, as shown in Table 1, which identifies the performance contribution of each sub-module in the IEM. It is worth mentioning that the local interactions bring more improvement than the global interactions. This may be due to the boundary assignment process that tends to capture discriminative features from the local region.
3.2 Applications
We verify the effectiveness of our proposed method on four challenging visual tasks, i.e., Camouflaged Object Detection (COD) [11], Polyp Segmentation (PS) [12], and Lung Infection Segmentation (LIS) [13]. The COD task mainly manifests in complex background/edges, small targets/structures, slender trunks/limbs, and partially occluded objects. The main feature of the PS task is its high diversity of size and texture. The main difficulty of the LIS task lies in the complex structures of the foreground. In addition, we also verify the effectiveness of our proposed method on the non-challenging figure- ground task.
Camouflaged Object Detection. Datasets: We evaluate our model on three challenging visual Camouflaged Object Detection datasets, i.e., CHAMELEON (CHA) [59]. CAMO (CAM) [27], and CODIOK (COD) [11]. CHA is collected from the Internet by searching "camouflaged animal" as the keyword. CAM is the first publicly released COD dataset. COD is the most challenging dataset with higher image quality. Implementation Details: We implement our model with PyTorch, and TITAN Xp GPUs are used for training and testing. ResNet-50 [17] is used as encoder in our model, which is initialized as the pre-trained weights on ImageNet. In this paper, we use the resolution of 480 480 during both the training phase and testing phase. Our model is trained for 100 epochs using the Adam
7

---

Table 2: Comparison with SOTA methods on
the CHA [59], CAM [27], and COD [11] datasets.
t indicates higher is better.
Table Comparison with four SOTA meth- ods on Kvasir, CVC-612, ColonDB, ETIS,
[78] [11] [10]
[70]
[38] [69]
Ours
and Endo datasets.
CAM 127 CHA 1591
S(1)
.848 .869 .888
.893
.882
.888
.902
E(t)
.870
.891 .942
.923
.942
.988
.947
[50] [82]
[14]
[12]
[80]
Ours
F())
.702
.740 .816
.813
.810
.796
.840
Dice(1)
.888 821 .723
.898
.907
.91T
M(J)
.050
.044 .030
.030
.033
.031
.030
Kvasir [18]
IoU(t)
.746 .743 .611 .840
.862
.858
S(T)
.322
.751 .820
.775
.782
.785
.803
F(t)
.794
.808 .670 .885 .893
.898
B(T)
.768
.771 .882
.847
.852
.859
.871
S(T)
.858
.862 .782 .915
.922
.922
COD [
F())
5583
.606
.743
.673
.695
.686
.748
E" (1)
.893
.910 .849
.948
.944
.953
M(J)
.104
.100
.070
.088
.085
.086
.068
M(1)
.055
.048 .075
.030
.028
.025
5(1)
.727
.771 .815
.814
.800
.888
.821
[58) ColonDB [60] CVC-612
Dice
.823
.794 .700
.899
.921
.924
B(T)
.779
.006
.887
.865
.868
.850
.895
(t)
.755
.729 .607
.849
.879
.884
F())
.509
.551 .680
.666
.660
.667
.687
F())
.811 .785 .6117
.896
.914
.930
M(J)
.056 .051 .037
.035
.040 .035
.031
S(1)
.889
.873 .793
.936
.911
.943
ETh
.954
.931 .885
.979
.972
.982
Table 3: Comparison with six SOTA methods on
M
.019
.022
.042
.009
.008
.008
the COVID-19 CT segmentation dataset.
(T)
.512
183
.199
.709
.755
.768
Dice(1) (1n.(1) Spec.(T) S(*) E(t) M[)
[o
.44
.410
.347
.640
.678
.683
F F())
.498
.467
.379
.696
.737
.746
[50]
.439
.534
.858
.622
.625 .186
[41]
.583
.637
.921
.744
.625
.112
S(1)
.712
.691
.634
.819
.836
.842
ET
.776
.760
.765
.869
.883
.868
[54]
.233
.658
.926
.725
.739
.102
M(
.061
.064
.094
.045
.041
.040
[29]
.515
.594
.840
.655
.814
.184
[82]
.581
.672
.902
.722
.662
.120
Dice
.398
.401 .297
.628
.719
.723
(oU(t)
.335
.344 .217
.567
.664
.651
[13]
.682
.692
.943
.781
.720
.082
F(t)
.366
.300 .231
.600
.678
.680
[20]
.700
.751
.860
.084
S(T)
.684
.683 .557
.794
.840
.822
Ours
.754
.748
.973
.799
.911
.056
ET
.740
.776 .633
.841
.830
.834
Table 4: Performance on DUTS-Test [65] and
M
.036
.035 .109
.031
.020
.015
PASCAL-S [30].
Dice
.710 .707 .467
.871
.869
.889
Endo [6]]
IoU(t)
.627 .624 .329
.797
.807
.817
Test
PASCAL-S
F(t)
.684
.687 .341
.843
.849
.865
M(1)
F(1) S(1) E(t)
M(1)
[(T) S(1) E(t)
S(T)
.843 .839 .640
.925
.925
.929
[79]
.041 .807 .885 .914
.062
.800 .858 .891
E (T)
.866
.898
.817
.972
.943
.978
[67]
.035 .840 .802 .927
.062
.825 .862 .901
M()
.022
.018
.065
.010
.010
.007
[28]
.032
.666 .899 .937
.061
.824 .863 .903
Ours
.033
.868 .902
.940
.940
.061
.82.887888 .866 .907
[23] optimizer with an initial learning rate of 0.0001, decreased by 0.1 at 50 epochs. The batch size is
32. Evaluation Criteria: S-measure (5.) ) [9], mean E-measure (E&) [37], weighted F-measure (F))
[37], and Mean Absolute Error (MAE) [42] are used as the evaluation metrics. Comparison with SOTA: We compare the performance of FGA-Net with SOTA methods, including EGNet [78), SINet [11], SINetv2 [10], MGL [70], PFNet [38], and UGTR [69]. Quantitative results are listed Table 2. Obviously, model outperforms the contenders. In particular, FGA-Net achieves significant performance improvement compared to the second-best method SINetv2 [10] on the challenging CODI0K dataset. It proves that with the help of Figure-Ground Ground assignment mechanism, it can make the network more efficient in solving the problem of separating ambiguous regions, which is caused by camouflaged objects that are highly similar to background in appearance.
Polyp Segmentation. Datasets: We evaluate our model on five challenging Polyp Segmentation (PS) datasets, Kvasir (Kvasir) [18], CVC-CCinicDB/CCCC-.
(CVC-612) [2]. CVC-ColonDB (ColonDB) [60], ETIS (ETIS) [58], and EndoScene (Endo) [61]. Kvasir is the most challenging dataset, which contains 1,000 images. CVC-612 dataset includes 612 open-access images from 31 colonoscopy clips. ColonDB and ETIS consist of 380 and 196 polyp images. Note that the training set only contains the data of Kvasir and CVC-612. We follow [12] to split all datasets. Implementation Details: We use the same implementation method as COD task, except that the batch size is 16. We use the resolution of 352 x 352 during both the training phase and testing phase. Evaluation Criteria: Dice [19], IoU [18], Fg' Sa. maximum E-measure (ET))), ), and MAE are used as the metries. Comparison with SOTA: We compare our FGA-Net with four SOTA medical
image segmentation methods, including UNet [50], UNet++ [82], SFA [14], Pra [12], and MSNet
8

---

[80]. Quantitative results are listed in Table 5. We can observe that our FGA-Net surpasses the aforementioned methods in most of the metrics. Compared to the second-bess methods on each dataset, FGA-Net outperforms all of them on five datasets. Similar results can also be observed in terms of the MAE score.
Lung Infection Segmentation. Datasets: We also evaluate our model for Lung Infection Segmenta- tion (LIS) on the COVID-19 CT segmentation dataset, including 100 axial CT images from different COVID-19 patients. We follow [13] to randomly select 45 CT images as training samples, 5 images for validation, and 50 images for testing. Implementation details: We use the same implementation method as COD task, except that the batch size is 24. We use the resolution of 352 x 352 during both the training phase and testing phase. Evaluation Criteria: Dice, Sensitivity (Sen.) [56]. Specificity
(Spec.) [57], Sa. Eygan, and MAE are used as the metrics. Comparison with SOTA: We compare our FGA-Net with six SOTA medical image segmentation methods including UNet [50], UN++ [82], AUN [41], GUN [54], DUN [29]. Inf [13], and ERRNet [20]. Quantitative results are listed in Table 3. We can observe that our model consistently outperforms all other contenders across all metrics,
Non- Mon-Challenging Figure-Ground Task. In Table 4, conduct experiments more non- challenging Figure-Groud Ground datasets (salient object detection), and the results show that the FGA-Net proposed in our paper is also competent for salient object detection. It indicates that FGA helps for different segmentation tasks by improving the boundary assignment capabilitt of the model, which in turn improves the final performance.
4 Related Work
Historically, the visual phenomenon most closely associated with the perceptual organization is grouping. and Figure-Grooe Groure assignment [24, 25, 53, 66]. In general, grouping determines what the qualitative elements of perception are, and figure-ground assignment determines the interpretation of those elements in terms of their shapes and relative locations in the layout of surfaces in the real-world [63, 64]. Figure-Ground assignment refers to the perceptual process of assigning a boundary separating two regions to one of them [52] and provides important configural prior for scene perception. Accordingly, the studies of the Figure-Ground assignment process [43, 63, 64] show that plays central role in aiding higher levels of visual perception.
Therefore, we hope to introduce the Figure-Ground assignment process into the segmentation model to help address the object segmentation. Factors that affect Figure-Ground assignment include size, surroundedness, orientation and contrast [52], symmetry [15], parallelism [39], convexity [21, 34, 39, 44], meaningfulness [43], and lower region [62]. In this paper, we mainly consider the cues related to the configuration, i... convexity and lower region. Compared to recent research [35, 36] applying the Figure-Ground Ground Assignment mechanism in some vision tasks without considering high-level supervisions, our work instead explores an end-to-end figure-ground ground aided approach that can be easily incorporated into existing incoer-decoded models and improve their performance.
5 Conclusion
In this paper, we demonstrate that the human vision can be effectively utilized improve the performance of CNN model for different challenging robust perception organization tasks, e.g., camouflaged object detection, polyp segmenta- tion, and lung infection segmentation. Specifically, we investigate the proposed FGA module, on a carefully established synthetic dataset for the rngure-Ground Segregation test, which aims to measure the capability of Figure-Ground Ground Assignment. The empirical study shows that the proposed new module consistently improves the performance of several representative networks in reducing visual ambiguities. Moreover, the model implemented via our proposed FGA mechanism outperforms SOTA approaches in three challenging real-world applications. We believe this study provides valuable insights to the community and attracts attention to exploring the gggure principles for other visual perception tasks.
cues inspired by the perceptual organization of
Societal Impact. Perceptual organization is one of the most challenging computer vision tasks. Albeit being challenging, Perceptual organization is beneficial to wide range of applications [75, 83). However, there also exists the risk that the technology is utilized the scenario of the illegal shoot, malicious edit, and incorrect use.

---

Acknowledgement This work was supported by National Key R&D Program of China under Grant 2020AAA0105701. National Natural Science Foundation of China (NSFC) under Grants 62225207, 61872327 and U19B2038, and the University Synergy Innovation Program of Anhui Province under Grants GXXT-2019-025.
References
[1] I. Bello. Lambdanetworks: Modeling long-range interactions without attention. arXiv preprint arXiv:2102. 08602, 2021.
[2] J. Bernal, F. J. Sánchez, G. Fermández-Esparrachh D. Gil, C. Rodríguez, and F. Vilariño. Wm-dova maps for
accurate polyp highlighting colonoscopy: Validation vs. saliency maps from physicians. Computerized
Medical Imaging and Graphics. 2015.
[3] E. Brunswik and J. Kamiya. Ecological cue-validity of "proximity" and of other gestalt factors. The
American Journal of Psychology, 66(1 ):20-32, 1953.
[4] K. R. Castleman. Digital image processing. 1993.
[5] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and L.. Yuille. Deeplab: Semantic image
segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEFE transactions
pattern analysis and machine intelligence, 2017.
[6] L.-C. Chen, G. Papandreou. F. Schroff, and H. Adam. Rethinking atrous convolution for semantic image
segmentation. arXiv preprint arXiv: 05588, 2017.
[7] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3606-3613, 2014.
[8] M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: retrospective. International journal of computer vision. 111(1):98-) 2015. [9] D.PP. Fan, M.-M. Cheng. Y. Liu, T. Li, and A. Borji. Structure-measure
new way to evaluate foreground maps. In IEEE International Conference on Computer Vixion (ICCV), 2017.
[10] D.PP. Fan, G.-P. Ji, M.-M. Cheng, and L. Shao. Concealed object detection. IEEE transactions on pattern
analysis and machine intelligence, 2021.
[11] D.PP. Fan, G.-P. Ji, G. Sun, M.-M. Cheng, J. Shen, and Shao. Camouflaged object detection. In IEFE
Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[12] D.-P. Fan, G.-P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao. Pranct: Paralicl reverse attention network for polyp segmentation. 2020.
[13] D.-. Fan, T. Zhou, G.P. Ji, Y. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao. Inf-net: Automatic covid-19
lung infection segmentation from et images. IEEE transactions medical imaging, 2020.
[14] Y. Fang, C. Chen, Y. Yuan, and K.-y. Tong. Selective feature aggregation network with area-boundary constraints for polyp segmentation. In International Conference on Medical Image Computing and Computer-Assistel Intervention. Springer, 2019.
[15] C. C. Fowlkes, D. R. Martin, and J. Malik. Local figure-ground cues are valid for natural images. Journal of Vision, 2007
[16] C. C. Fowlkes, D. R. Martin, and Malik. Local figure-ground cues are valid for natural images. Journal
of Vision 7(8), 2007.
[17] K. He, Zhang. Ren, and Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
[18] D. Jha, H. Smedsrud, M. Riegler, P. Halvorsen, T. Lange, Johansen, and H. D. Johansen. Kvasir-seg: A segmented polyp dataset. In International Conference on Multimedia Modeling. Springer,
2020.
[19] D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, and H. D. Johansen. Resunet++: An advanced architecture for medical image segmentation. In 2019 IEEF International Symposium on Multimedio (ISM). TEFF, 2019.
[20] G.PP. Ji. L. Zhu, M. Zhuge, and K. Fu. Fast camouflaged object detection via edge-based reversible re-calibration network. Puttern Recognition, page 108414, 2021.
[21]
G. Kanizsa. Convexity and symmetry figure-ground organization. Vision and artifact, 1976.
[22] J. Kim, D. Linsley, K. Thakkar, and T. Serre. Disentangling neural mechanisms for perceptual grouping.
arXiv preprint arXiv:1906 015.0, 2019.
[23]
D. Kingma and Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988
2014.
[24]
K. Koffka. Perception: an introduction to the gestalt-theorie, Psychological Bulletin, 19(10):531, 1922.
[25]
W. Köhler. An aspect of gestalt psychology. The Pedagogical Seminary and Journal of Genetic Psychology,
32(4):611-723, 192..
[26] V. A. Lamme. The neurophysiology of figure-ground segregation in primary visual cortex. Journal of neuroscience. 15(2):1605-1615 1995.
[27] T.-N. Le, T. V. Nguyen, Z. Nie, M.-T. Tran, and A. Sugimoto. Anabranch network for camouflaged object segmentation. Computer Vision and Image Understanding (CVIU), 2019.
[28] A. Li, J. Zhang, Y. Lv, B. Liu, T. Zhang, and Y. Dai. Uncertainty- aware joint salient object and camouflaged object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 10071-1008 2021.
10

---

[29] X. Li, H. Chen. X. Qi. Q. Dou, C.-W. Fu, and P.-A. Heng. H-ddnseunee: hybrid densely connected unet for
liver and tumor segmentation from et volumes. IEEE transactions medical imaging, 2018.
[30] Y. Li, X. Hou, C. Koch, J. M. Rehg, and A. L. Yuille. The secrets of salient object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 280-287, 2014.
[31] D. Linsley, J. Kim. V. Veerabadran, and T. Seree Learning long-range spatial dependencies with horizontal
gated-rreurrent units. arXiv preprint wwXiv:885,.05. 08310881..22 2018.
[32] D. Linslcy, D. Shiebler, S. Eberhardt, and T. Serre. Learning what and where attend. arXiv preprint arXiv: 1805. 088/9, 2018.
[33] Long. E. Shelhamer, and T. Darrell, Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015. [34] Y. Lu, W. Zhang, H. Lu, and X. Xue. Salient object detection using concavity context. In 201/ International
Conference on Computer Vision. IEFE, 2011 1.
[35] M. Maire. Simultaneous segmentation and figure/ground organization using angular embedding. In
European Conference on Computer Vision, pages 450-464, Springer, 2010.
[36] M. Maire, T. Narihira, and S. X. Yu. Affinity cnn: Learning pixel-centric pairwise relations for figure/ground embedding. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages
174-182, 2016.
[37] R. Margolin, I.. Zelnik-Manor, and A. Tal. How to evaluate foreground maps? In IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[38] H. Mei. G. Z. Z. Wei, X. Yang. X. Wei, and D.-P. Fan. Camouflaged object segmentation with distraction mining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8772-8788 2021.
[39]
W. Metzger. Gesetze des sehens [laws of vision]. Frankfurt: Waldemar Kramer, 1953.
[40]
K. Murphy, Machine learning: a probubilistic perspective. MIT press. 2012.
[41]
O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y.
Hammerla, B. Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint
1rXivv8804.09999 [42] F. Perazzi, P. Kräthnbühhl, Y. Pritch, and A. Hornung. Saliency filters: Contrast based filtering for salient region detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
2018.
[43] M. A. Peterson. Object recognition processes can and do operate before figure- ground organization.
Current Directions in Psychological Science, 1994.
[44] M. A. Peterson and E. Salvagio. Inhibitory competition in figure-ground perception: Context and convexity.
Journal of Vision, 2008
[45] I. Pitas. Digital image processing algorithms and applications. John Wiley & Sons, 2000.
[46] J. Poort, F. Raudies. A. Wannig. V. A. Lamme, H. Neumann, and P. R. Roelfsema. The role of attention in
figure-ground segregation in areas v1 and v4 of the visual cortex. Neuron, (5(1):143-156 2012.
[47] F. T. Qiu and R. Von Der Heydt. Figure and ground in the visual cortex: V2 combines stercoscopic cues with gestalt rules. Neuron, 2005.
[48] S. Ren, K. He. R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region
proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 2015. [49] P. R. Roelfsema, V. A. Lamme, H. Spekreijse, and H. Bosch. Figure-ground segregation in recurrent network architecture. Journal of cognitive neuroscience, 14(4):525-$37, 2002.
[50] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation.
In International Conference on Medical image computing and cmmpueer-assisted intervention. Springer,
2015.
[51] F. Rubin. Visuell wahrgenommene figuren: Studien in psychologischer analyse, volume 1. Gyldendalske
boghandel. 1921.
[52]
E. Rubin. Visuell wahrgemonmene figuren. Ripol Classic, 1980.
[53] F. Sander. Structure, totality of experience, and gestalt. 1930.
[54] J. Schlemper, O. Oktay, M. Schaap. M. Heinrich, B. Kainz, B. Glocker, and D. Rueckert, Attention gated
networks: Learning to leverage salient regions in medical images. Medical image analysis, 53:197-200 2077-07,
2019.
[55] M. W. Self, T. van Kerkoerle, H. Supèr, and P. R. Roelfsema. Distinct roles of the cortical layers of area v1 in figure-ground segregation. Current Biology. 23(21):2121-2112, 2013.
2013.
[56] F. Shan, Y. Gao, J. Wang. W. Shi, N. Shi, M. Han, Z. Xue, and Y. Shi. Lung infection quantification of covid-19 19 in et images with deep learning. arXiv preprint arXiv:2003. 2003006635455 04655, 2020.
[57] F. Shi, L. Xia, F. Shan, D. Wu, Y. Wei, H. Yuan, H. Jiang. Y. Gao, H. Sui, and D. Shen. Large-scale screening of covid-19 -19 from community acquired pneumonia using infection size-aware classification. arXiv preprint Xix:200003 (9860, 2020.
[58] J. Silva, A. Histace, O. Romain, X. Dray, and B. Granado. Toward embedded detection of polyps in wee images for early diagnosis of colorectal cancer. International Journal of Computer Assisted Radiology and Surgery, 2014.
[59] P. Skurowski, H. Abdulameer, J. Blaszczyk, T. Depta, A. Kornacki, and P. Kozict. Animal camouflage analysis: Chameleon database. Unpublished Manuscript, 2018.
[60] N. Tajhakhsh, S. R. Gurudu, and J. Liang. Automated polyp detection in colonoscopy videos using shape and context information. IEEE transactions on medical imaging, 2015.
[61] D. Vázquez, J. Bernal, F. J. Sánchez, G.
A. M. López, A. Romero, M. Drozdzal, and A. Courville, A benehmark for endoluminal scene segmentation of colonoscopy images. Journal of
11

---

healthcare engineering, 2017.
[62] S. P. Vecera, E. K. Vogel, and G. F. Woodman. Lower region: A new cue for figure- figure-ground assignment.
Journal of Experimental Psyckology: General, 2002.
[63] J. Wagemans, J. H. Elder, M. Kubovy, S. E. Palmer, M. A. Peterson, M. Singh, and R. von der Heydt. A century of gestalt psychology in visual perception: perceptual grouping and figure-ground ground organization.
Psychological bulletin, 138(6):1172. 2012.
[64] J. Wagemans, J. Feldman, S. Gepshtein,
Van der Helm, and C. Van Leeuwen. A century of gestalt psychology in visual perception: li. conceptual and theoreti- cal Toundations. Psychological bulletin. 138(6):12188 2012.
Kimchi, J. R. Pomerantz,
[65] L. Wang. H. Lu, Y. Wang. M. Feng. D. Wang. B. Yin, and X. Ruan. Learning to detect salient objects with image-level supervision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 136-145, 2017.
[66]
M. Wertheimer. Untersuchungen zur lehre von der gestalt. Psychologische forschung, 1(1):47-58, 1922. Z. Wu, L. Su, and Q. Huang. Decomposition and completion network for salient object detection. IEEE
[67]
Transactions on Image Processing, 30:6226-622 6239, 2021.
[68] Y. Xu, Q. Zhang, J. Zhang, and D. Tao. Vitae: Vision transformer advanced by exploring intrinsic inductive
bias. Advances in Neural Information Processing Systems, 34:28522-22535,, 2021.
[69] F. Yang. Q Zhai, X. Li, R. Huang, A. Luo, H. Cheng, and D.-P. Fan. Uncertainty-gut
transformer reasoning for camouflaged object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4146-4155, 2021.
[70] Q. Zhai, Li, F. Yang. C. Chen, H. Cheng, and D.P. Fan. Mutual graph learning for camouflaged object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12997-13007 2021.
[71] W. Zhai, Cao, H. Xie, and 7.-J. Zha. Deep texton-coherence network for camouftaged ohject detection.
IEEF Transactions on Multimedia, 2022.
[72] W. Zhai, Y. Cao, Z.-. Zha, H. Xie, and F. Wu. Deep structure-revealed network for texture recognition.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11010-
11019, 2020.
[73] W. Zhai, Y. Cao, J. Zhang, and Z.-J. Zha. Deep mulippleattribbte--erreiiee network for real-world texture recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
3613-3622, 2019.
[74] W. Zhai, J. 7hu, Y. Cao, and 7., Wang. A generative adversarial network based framework for unsupervised visual surface inspection. In 2018 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP). IEEE, 2018
[75] W. Zhai, J.Zh, Y. Cao, and Z. Wang. A generative adversarial network based framework for unsupervised visual surface inspection. In 2018 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 1283 1287, IEEE, 2018.
[76] H. Zhang, K. Dana, J. Shi, 7. Zhang, X. Wang, A. Tyagi, and A. Agrawal. Context encoding for semantic segmentation. In Proceedings of the IEEE conference on Computer Vision and Puttern Recognition, pages
7151-7160, 2018
[77] J. Zhang and D. Tao. Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things. IEEE Internet of Things Journal, 8(10):7789773 2020. [78] J. Zhao, J. Liu, D. Fan. Y. Cao, J. Yang, and M.-M. Cheng. Egnet: Edge guidance network for salient object detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[79] x. Zhao, Y. Pang, I.. Zhang, H. Lu,, and I.. Zhang. Suppress and halance: A simple gated network for salient object detection. In Purppen Conference on Computer Vision, pages 35-51. Springer, 2020.
[80] X. Zhao, L. Zhang, and H. Lu. Automatic polyp segmentation via multi-scale subtraction network.
In International Conference on Medical Image Computing and Computer- Computer-Assisted Intervention, pages
120- 130. Springer, 2021.
[81] H. Zhou, H. S. Friedman, and R. Von Der Heydt. Coding of border ownership in monkey visual cortex.
Journal of Neuroscience, 2000.
[82] 7. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and Liang. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, 2018.
[83] K. Zhu, W. Zhai, Z.J. Zha, and Y. Cao. Self-supervised tuning for few-shot segmentation. arXiv preprint arXiv:2004.0533 2020.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Abstract and Section
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 5.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See supplemen- tal material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 3.1 and Section 3.2.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 3.2.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Section 3.1 and Section 3.2
(b) Did you mention the license of the assets? [No] All of the datasets used are publicly available for research.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you's 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]


Reviews:
[{"review_id": "shp0GNjPmD", "reviewer": "Reviewer_fHq8", "summary": "This paper focuses on perceptual organizatio where the goal is to perceive and group the individual visual element. To address the visual ambiguity in the discrimination process, authors propose to use e figure-ground assignment mechanism to improve deep neural networks  to achieve a robust perceptual organization. Specifically, a Figure-Ground-Aided (FGA) module is presented to learn the configural statistics of the visual scene and leverage it for the reduction of visual ambiguity. An Interactive Enhancement Module (IEM) is designed to leverage configural cues from FGA to enhance the representation learning. Experiments are conducted on four applications.", "questions": "Please refer to the weakness part in theStrengths And Weaknesses section.", "limitations": "Authors discussed the limitations of the method in the main paper and the appendix. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\n+ The idea of taking the inspiration from the perceptual organization of human vision and studying the figure-Ground cues is interesting. The motivation is also well explained. \n\n+ Experiments are conducted on different robust perception organization tasks, such as camouflaged object detection, polyp segmentation, and lung infection segmentationm. The results validate that the proposed method can help improve the performance of robust perception organization tasks. \n\n+ The proposed Figure-Ground Segregation test can help to investigate the performance of models in Figure-Ground assignment.\n\n\nWeakness:\n\n- The performance gaps between the proposed method and existing methods are marginal in some metrics. The best numbers are not always marked in bold which is missleading, e.g. second row of \"Figure 9\", second row of \"Figure 10\".\n\n- The main experimental result tables are titled as Figures, such as Figure7, 8, 9, 10. To me, they are tables rather than fingures. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "hyNGpKKinTr", "reviewer": "Reviewer_dfqS", "summary": "The paper proposes a module for enhancing figure-ground assignment, and which can be used to extend existing encoder-decoder based segmentation models. More specifically, the authors implement mechanisms for measuring two figure-ground cues, namely ‘convexity’ and ‘lower region’. These two cues come from the cognitive vision literature, implemented via morphological operators, and inserted into the encode-decoder CNN architecture. Experimental evaluation of known CNNs augmented with this novel module are then performed on challenging segmentation tasks, e.g., those of medical imaging, indicating an improvement of accuracy in most tasks.", "questions": "The IEM is built upon adding more learning units through the spatial self-attention mechanism. I wonder if adding parameters, and/or depth can improve the IoU and S-measure on the FG test set. Can you comment on the number of parameters added by the IEM units to the ResNet50, and speculate about performance of deeper segmentation models?", "limitations": "It will be good to exemplify few failure cases of your model (e.g., on the FG or medical datasets). Perhaps other FG factors are needed? (e.g., good continuation?). ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n•\tFigure-ground assignment is indeed a fundamental process for computer vision algorithms, which is yet not fully explored. I like the idea of adding FG mechanisms to the current FCN segmentation models.\n•\tThe FG segregation test is an important test, which is indeed popular on the human vision literature. I agree that segmentation models such as DeeplabV3 should be able to replicate human behavior in this test set, with and without additional training from a similar set. \n\nWeaknesses:\n•\tThe paper is a bit hard to follow, and several sections were needed more than one reading pass. I suggest improving the structure (introduction->method->experiments), and put more focus on the IEM in Fig 3, which is in my view the main figure in this paper. Also, to improve the visualization of the Fig 7, and Fig. 10. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "IgdyH_89gVq", "reviewer": "Reviewer_ghhU", "summary": "The paper proposes a Figure-Ground-Aided (FGA) module to improve the performance of existing CNN models in figure-ground segregation. In particular, the method considers two figure-ground assignment cues, i.e., convexity and lower-region, and derives two supervisory signals automatically from ground-truths for network supervision. An interactive enhancement module is proposed to enhance feature representations. The results look good.", "questions": "- I am particularly concerning  whether the lower-region cue is well connected to the challenge studied in the work. More discussions are demanded. In addition, I am wondering how will the model perform by directly using a edge-aware loss like HED [ref3] instead of $\\mathcal{L}_{LR}$.\n\n- I am unsure about the meaning of \"stronger supervisory signals\" in Line 89. Why is it a weak form by \"directly using mask labels as supervision\"? This is not convinced to me since many competitors in Tables 7-10 only use ground-truths as supervision, and still obtains very promising performance. \n\n- The statement \"figure-ground assignment ... contributes almost all perception-based tasks\" seems an overclaim, or more discussions should be provided to support the argument. \n\n- So many errors in only seven formulas:\n  - Eq.1 and Eq.2 are not well written. What does the symbol $\\Leftarrow$ refer to? \n  - It is unclear the meaning fo $GT_{L_2}$ in Eq. 2.\n  - How are $K_c$ and $K_L$ determined? Do they set to the same values across the datasets? How will the different values affect the performance?\n  - L184 only gives the definition of $Q^{LR}$, however, strictly speaking, $Q^{Con}$ is not defined.\n  - It is not clear what the $E$ in Line 187 refers to. \n  - $\\mathcal{L}_M$ is not defined. Should it be $\\mathcal{L}_D$ in Line 193? \n  - It is also not clear to me how $M$ in Eq. 5 will be used in the network. \n  -  The $\\texttt{Lambda}$ in Eq. 3 is very similar to a self-attention operator. Why is $\\texttt{Lambda}$ used here? \n  - The symbols $Cue_{LR}$ and $Cue_{Con}$ shown in Fig. 3 are not consistent with symbols in Section 2.\n  - $\\{D_i\\}_{i=1}^5$ in Fig.3 are also not defined.\n  - What does the $P_d$ refer to? The symbols is potentially misused since it is defined to denote position embedding in Line 183.\n\n- Typos or grammar errors:\n  - \"fully connection layer\" should be \"fully connected layer\"\n  - \"k-th feature\" should be \"the k-th feature\"\n\n[ref3] Xie, Saining, and Zhuowen Tu. \"Holistically-nested edge detection.\" ICCV 2015.\n\n", "limitations": "No limitation is discussed. ", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nThis work is on addressing figure-ground segregation of situations with high visual ambiguity. The paper overall is well organized and easy to read in most sections (not in the technical part). Experiments on a synthetic benchmark and four practical applications are surely extensive and the results look good to me.\n\nWeaknesses\n\nWhile it makes sense to me to formulate perceptual theory into neural networks, the motivation to use the specific two cues (convexity and lower-region) instead of many other cues (size, surroundedness, orientation and contrast, symmetry, parallelism, meaningfulness) as mentioned in Related Work (Line 343) is not quite clear to me. The two cues explored here have also been explored previously, e.g., in [ref1,ref2], thus the novelty seems to be limited. Regarding the performance, even though that the authors argue that \"existing methods degrades significantly when deployed in cases with complex visual ambiguity\", I found from Tables 7-10 that the improvements are very minor in most datasets. Thus, I am concerning the core motivation of the method and the effectiveness of the technical implementations. Last, the formulations in Section 2 are not well presented. Though I can get the main point, the poor formulation makes it hard for me to understand all the details. My detailed questions are given in the next section.\n\n[ref1] Sundberg, Patrik, et al. \"Occlusion boundary detection and figure/ground assignment from optical flow.\" CVPR 2011\n\n[ref2] Lu, Yao, et al. \"Salient object detection using concavity context.\" ICCV 2011.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7hfkIet67f", "reviewer": "Reviewer_cfAi", "summary": "This work proposes a representation learning method to achieve the Perceptual Organization process implicitly. The authors design a new module named as Figure-Ground-Aided (FGA), in which the convexity and lower region cues are leveraged by the Interactive Enhancement Module (IEM). The model takes traditional CNNs as the backbone. The optimization utilizes segmentation labels and binary cross-entropy (BCE) loss. In the optimization, the convexity and lower region cues derived by segmentation labels are also used as the supervision. Experiments based on the synthetic and realistic datasets are designed to verify the effectiveness. Overall, the motivation of this paper is clear and the designed model is technically sound. Moreover, some claims of this paper are not factual.", "questions": "1. Why authors design the experiments on the synthetic dataset? If the realistic datasets may include some other irrelevant factors for Figure-Ground Segregation, authors should explain this problem explicitly.\n2. There are some mistakes in the notations and expressions:\n(1) In Eq. (7), the notation L_M has no explanation. \n(2) In line 277, authors state that “We verify the effectiveness of our proposed method on four challenging visual tasks”. However, the following statement only mention three tasks (COD, PS, LIS).\n(3) In Figure 7, authors state that “Comparison with 10 SOTA methods ...”. However, only six comparison methods are involved.\n(4) In the second line of Figure 10, the best result is not 0.858 but 0.862. The bold style is marked incorrectly. Authors should check this kind of mistake carefully.\n(5) In Figure 9, the results of [66] on two different datasets are identical. Please check it.\n(6) In line 148, the subscript i has no explanation. \n(7) In line 193, the notations “i\" and “I” has no explanation. \n(8) Figures 7, 8, 9 and 10 should be marked as tables.\n", "limitations": "The authors give the societal impact in Section 6. However, there is no limitations discussed in this section. The reviewer believes that the requirement of segmentation labels should be a limitation. Specifically, the traditional object detection models only require box labels, which is cheaper than segmentation labels.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The originality is good. This work is inspired by cognitive researches. The motivation is clear and interesting, while the designed model is novel and technically sound.\n2. The paper is organized well and the statements are clear.\n3. The significance is enough. This work aims to learn visual representations being discriminative to the confusing objects and backgrounds. This is an essential problem in computer vision.\n\nWeaknesses:\n1. Some necessary implementation details are missing:\n   (1) The model architecture is not introduced completely. In Figure 3, the intermediate features E^Con_4, ..., E^Con_1 and E^LR_4, ..., E^LR_1 are derived from the feature E_5. However, there is no description for this process.\n   (2) The size of structure element K_C is 10 while that of K_L is 5. Why authors select these hyperparameters?\n   (3) In the ablation experiments, it is not clear how to implement the variant models (a), (b) and (c) without the IEM module.\n\n2. In the main paper, there is no direct evidence verifying that the proposed model can reducing the visual ambiguousness in realistic datasets. Although a large number of realistic datasets are used in the comparison experiments, only the overall quantitative results are shown in the paper. The improvements of these quantitative results may be unrelated to the reduction of visual ambiguousness. \n  Therefore, the ablation experiments on the realistic datasets are also necessary. Note that the experimental results on the synthetic datasets cannot be considered as a direct evidence for this problem.\n\n3. Some claims of this paper are not factual.\n  (1) In line 297, authors claim that “FGA-Net achieves significant performance improvement compared to the second-best method UGTR [67]”. However, UGTR [67] should not be the second-best method according to the results in Figure 7. For example, SINetv2 [10] is better than UGTR [67] for most of the metrics. Moreover, the superiority of the proposed model is not very significant compared with SINetv2 [10].\n  (2) In line 325, authors claim that “Quantitative results are listed in Table 8. We can observe that our model consistently outperforms all other contenders across all metrics”. However, the proposed model is inferior than [20] for the metric Sen.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept