Exploring Figure-Ground Assignment Mechanism in Perceptual Organization
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Perceptual organization is a challenging visual task that aims to perceive and groupthe individual visual element so that it is easy to understand the meaning of thescene as a whole.
Most recent methods building upon advanced Convolutional Neural Network (CNN) come from learning discriminative representation and modelingcontext hierarchically.
However, when the visual appearance difference betweenforeground and background is obscure, the performance of existing methods degrades signiﬁcantly due to the visual ambiguity in the discrimination process.
Inthis paper, we argue that the ﬁgure-ground assignment mechanism, which conformsto human vision cognitive theory, can be explored to empower CNN to achieve arobust perceptual organization despite visual ambiguity.
Speciﬁcally, we present anovel Figure-Ground-Aided (FGA) module to learn the conﬁgural statistics of thevisual scene and leverage it for the reduction of visual ambiguity.
Particularly, wedemonstrate the beneﬁt of using stronger supervisory signals by teaching (FGA)module to perceive conﬁgural cues, i.e., convexity and lower region, that humandeem important for the perceptual organization.
Furthermore, an Interactive Enhancement Module (IEM) is devised to leverage such conﬁgural priors to assistrepresentation learning, thereby achieving robust perception organization withcomplex visual ambiguities.
In addition, a well-founded visual segregation testis designed to validate the capability of the proposed FGA mechanism explicitly.
Comprehensive evaluation results demonstrate our proposed FGA mechanism caneffectively enhance the capability of perception organization on various baselinemodels.
Nevertheless, the model augmented via our proposed FGA mechanism alsooutperforms state-of-the-art approaches on four challenging real-world applications.
The source code will be made available to the public.
Introduction
Perceptual organization, which is a vital visual task, refers to the processes by which the disjointbits of visual information are structured into the larger coherent units that we eventually experienceas environmental objects.
Thanks to the developments of discriminative representation learningand hierarchical context modeling with convolutional neural networks [17, 33, 48], the past fewyears have witnessed tremendous progress in perceptual organization.
Recent advanced methods areusually based on the framework of fully convolutional network (FCN) [33, 71, 50, 5], which learna discriminative feature representation and hierarchically model the local context by supervisionderived from human-given labels.
However, the performance of existing methods degrades signiﬁcantly when deployed in somechallenging tasks with complex visual ambiguity, such as camouﬂaged object detection [11, 27],medical image segmentation [12, 50], and visual industrial detection [70].
Since the visual appearance differences between the foreground and background are obscure, it is difﬁcult to perceive thecorrelation between individual visual elements and determine the boundaries.
The visual ambiguitiesimpede CNN’s represent learning and contextual modeling, leading to inaccurate and incompleteperceptual organization.
[Figure]
To address this issue, moreperceptual knowledge is required to be incorporatedinto visual organization.
Inthe 1920 s, the Gestalt psychologists identiﬁed Grouping and Figure-Ground astwo important principlesunderlying the process ofperceptual organization [24,66, 53, 25].
Grouping principle refers to bring togetherindividual visual elementsthat produce stimuli to forma holistic perception.
It hasattracted more attention because it is intuitively similar to the contextual modeling ability of CNN.
Different from Grouping, FigureGround assignment refersto the perception that assigns a boundary separatingtwo regions to one of them[52].
By introducing theconﬁgural statistics of the natural world in which the visual system evolved [3], the Figure-Groundassignment principle is thought to be important in reducing the visual ambiguousness of a scene.
Inaddition, Cognitive science studies [15, 47, 75] have found that neural activity associated with the
Figure-Ground assignment mechanism in the V 2 cortex of human vision occurs as early as 10 - 25 msafter the generation of visual stimuli, providing strong support for the role of local bottom-up processing.
Another study [43] on the “meaningfulness principle” also showed that assigned ﬁgures tendto be associated with neighborhoods with familiar shapes, pointing to the integration of knowledgefrom the top-down.
Inspired by these studies, as shown in Fig.
1, we argue that exploring ﬁgure-ground assignmentmechanism can empower CNN the ability of perceptual organization despite visual ambiguity, andconsequently presents a novel Figure-Ground-Aided (FGA) module that learns the conﬁgural statisticsof visual scene, and leverages it for representation learning.
Speciﬁcally, we ﬁrstly investigate theconﬁgural cues related to the Figure-Ground assignment mechanism in human psychophysics and ﬁndthat the ﬁgural region usually takes on the shape instructed by the separating boundary and appearscloser to the viewer, while the ground region is seen as extending behind the ﬁgure [43].
Typically,as shown in Fig.
2 (a), Convexity cue [39, 21, 44, 34] corresponds to the regions on either side ofthe boundary where the scene depth changes abruptly and is beneﬁcial for analyzing the hierarchicalrelationship between neighboring regions in the image, facilitating hierarchical contextual modeling.
And lower region cues [62, 16] usually corresponds to the region of the scene where occlusion hasoccurred and is beneﬁcial for analyzing the occlusion relationship between various neighborhoods inan image and determining the shape attribution of foreground objects and background regions.
After that, instead of using a weak form of directly using mask labels as supervision, we refer tothe method in [32] of teaching the FGA module to perceive the conﬁgural cues that human deemimportant for the perceptual organization by using human-derived labels as stronger supervisorysignals.
Notably, the Ground-Truth (GT) segmentation masks imply the prior knowledge of theannotators’ understanding of natural scenes.
Therefore, we easily generate the labels of the twoconﬁgural cues from the Ground-Truth (GT) label without the intervention of additional informationand exploit them as supervisions for the FGA module, facilitating the modeling of the conﬁgural statistics of natural scenes.
Furthermore, an Interactive Enhancement Module (IEM) is presentedto progressively enhance the discriminative features for boundary assignment via a local/globalinteractive strategy.
Speciﬁcally, a collaborative local interaction process is ﬁrst introduced byswapping queries to align locally between contextual co-occurring conﬁgural features.
Then, a globalinteraction process is introduced to establish global spatial correlations of local conﬁgural features,resulting in complete object boundaries.
Moreover, a Lambda strategy is utilized to improve thecomputational efﬁciency and performance of the original self-attention module.
F
G G
F
G
F
L o w er
R e g i o n
C o n v e x i t y
F
G #1 # # # # # # #
GT
LR
Convexity
[Figure]
To systematically investigate the performance of our proposed method for the
Figure-Ground assignment
, we design asynthetic computer vision task inspiredby an important experiment in cognitivescience—Figure-Ground Segregation [26,46, 49, 55].
Furthermore, the comprehensive experiments demonstrate that our proposed FGA module can facilitate the CNNto learn more efﬁciently in the reduction ofvisual ambiguities with low data requirements.
Finally, we also validate the performance of our proposed mechanism in fourchallenging visual applications, includingcamouﬂaged object detection [11], polypdetection [12], and lung anomaly detection[13].
The results demonstrate the superiority of our method over SOTA methods.
Thecontributions are summarized as follows:(
This
paper explores the Figure-Ground assignment mechanism from human visioncognitive theory to empower CNN to learnconﬁgural statistics to reduce visual ambiguities, thereby achieving robust perceptual organization.
This paper presentsa novel Figure-Ground-Aided module tointegrate ﬁgure-ground-aided cues in a hierarchical manner.
An Interactive Enhancement Module is devised to progressivelyenhance the discriminative features for Figure-Ground assignment via a local/global interactivestrategy.
(3) This paper introduces the Figure-Ground Segregation test, a synthetic visual perceptionchallenge to systematically investigate the performance of models in Figure-Ground assignment.
Experimental results demonstrate our proposed FGA module consistently improves the Figure-Groundassignment performance of several representative networks on datasets of different difﬁculty levels.
(4) Extensive experiments are also performed on four challenging robust object segmentationapplications, showing that the model constructed via our FGA module outperforms SOTAs.
Methodology
Architecture.
In this paper, we propose a novel Figure-Ground-Aided (FGA) module that uses conﬁguration cues from the Figure-Ground assignment process to enhance the perception of foregroundbackground relationship, thereby achieving a robust perceptual organization result.
It can be easilyincorporated into existing encoder-decoder models and improve their performance.
Fig.
3 showsthe overall architecture of the proposed Figure-Ground-Aided module.
Our proposed FGA moduleis composed of two cue-aided branches and an Interactive Enhancement Module (IEM).
Here, weuse ResNet-50 [17] as the backbone.
Speciﬁcally, we remove the fully connected layer and retainall convolutional bocks.
Given an input image of shape H × W , this backbone will generate ﬁvescales of features with spatial resolution gradually decreasing by stride 2.
We denote these featuresas E
= { E k | k
= 1 , 2 , 3 , 4 , 5 } .
The size of the k -th feature is (cid:2) C k , H 2 k , W 2 k (cid:3) , where C k is the numberof channels of the k -th feature.
We utilize the feature from { E k | k
= 1 , 2 , 3 , 4 } as skip-connected
E ncode r
Decoder E
E
E E
E
D 2 D 3 D 4 D
D
Supervision
Side Output
Skip Connection
C Concatenation
Top-Down information Flow
Bottom-Up information Flow
M
E Con
E LR
E
Lambda
Q Q
Lambda Lambda
Collaboration
Y
E G V
K
V K E LR 1 E LR
E Con 1 E Con
E LR 2 E LR
E Con 2 E Con
E Con 3 E Con E Con 4 E Con
E LR 3 E LR 3 E LR 4 E LR
Cue L Cue L Cue C [Figure] C
Loss
Input
GT
Generate Figure-Ground Cues
U p s a m p l e C
Aggregation Module U p s a m p l e C
Aggregation Module
G l oba l I n t e r ac t i on
G l oba l I n t e r ac t i on
Interactive Enhancement Module IEM Interactive Enhancement Module IEM Interactive Enhancement Module IEM
C o ll abo r a t
i v e
Loca l
I n t e r ac t i on C o ll abo r a t i v e
Loca l I n t e r ac t i on M We leverage the convexity and lower region cues, which can be calculated directly from the GT label and do not require any additional information, as supervisions for perceiving the contextual information of object boundary, respectively.
We ﬁrst allow the network to learn the statistical features related to the conﬁguration in a top-down manner. Then we use the learned features to guide the bottom-up features from the encoder to integrate bottom-up and top-down information. Interactive Enhancement Module (IEM) , which consists of two parts: Collaborative Local Interaction (CLI) and Global Interaction (GI).
IEM has three input features ( E Con , E LR , and E ), two of which come from the intermediate feature learned by the Figure-Ground assignment cues aided branches ( E Con and E LR ).
“Lambda” indicates the use of a Lambda strategy [1] to improve the computation efﬁciency of the CLI and GI interaction. feature for FGA module and the decoder. D =
{ D k | k = 1 , 2 , 3 , 4 , 5 } are the feature maps ofthe decoder part. The intermediate features E Conk and E LRk are derived from the feature E 5 . Thecalculation process here is the same as in the Decoder part.
Accordingly, we propose to introduce the convexity and lower region cues, as shown in Fig. 2 (b).
Togenerate the convexity cue, we used morphological opening and closing operations [45, 4].
It can beformed as
Cue C =
Cat ( Cue C 1 , Cue C 2 )
, “ Cat ” is the concatenation operation.
[Equation]
where GT is the ground truth segmentation mask.
Φ ◦ ( . ) is the opening operation.
Φ • ( . ) is the closingoperation.
K C is the structure element whose size is 10.
We get the convexity Cue by concatenating
Cue C 1 and Cue C 2 , whose shape is [2 , H, W ] .
H and W are the height and width of the mask.
Togenerate the lower region cue, we used morphological erosion and expansion operations.
It can beexpressed as: Cue
[Equation]
[Equation]
where Φ (cid:9) ( . ) is an erosion operation.
Φ ⊕ ( . ) is the expansion operation.
K L is the structure elementwhose size is 5.
We get lower region cue by concatenating Cue L 1 and Cue L 2 .
Interactive Enhancement Module.
To effectively utilize the important conﬁgural cues (convexityand lower region) provided by the Figure-Ground assignment mechanism, a novel Interactive En- hancement Module (IEM) is devised.
The IEM has three input features, two of which come from theintermediate feature learned by the Figure-Ground assignment cue-aided branches.
We deﬁne themas E Con and E LR , respectively.
Moreover, the remaining branch is the skip-connection feature Efrom the encoder.
We further improve the representation ability of the encoder feature by establishingthe interactions among the features of the skip-connection features and aided branches feature.
Asshown in Fig.
3, the interactions in the IEM consist of collaborative local interaction and globalinteraction.
1) Collaborative Local (CLI) interaction: the local interaction of the two Figure-Groundcues integrates the local structural context features.
Global interaction (GI): the global interactionbetween base features and local contextual features aims to enhance foreground features and suppressthe background with structural context features.
The detailed structure of the Interactive Enhancement Module is illustrated in Fig.
We use aspatial self-attention mechanism that considers both content and location interactions to achievelocal or global interaction of features.
This paper uses an efﬁcient Lambda layer [1], which capturesinteractions of feature elements by transforming available contexts into linear functions, termedlambdas, and applying these linear functions to each input separately.
For the collaborative localinteraction part, it can be expressed as follows:
[Equation]
where Y is the output of CLI.
P denotes the relative position embeddings.
Q LR is the query, which iscomputed by: Q LR = E LR W LR Q .
Q Con is calculated in the same way.
W LR Q is the learnable weights.
Lambda ( . ) is deﬁned as:
[Equation]
where the range of relative position in CLI is set to 5.
Cue ∈ LR, Con
For the global interaction part,it can be expressed as follows:
[Equation]
where E G =
Cat ( Y, E ) .
E G and M correspond to the feature maps shown on the right side of the IEM in Fig.
The Lambda ( . ) function is deﬁned as follows:
[Equation]
where the range of relative position in GI is set to the whole feature map size.
Loss Function.
In this section, we introduce the loss function used to train FGA-Net.
We adopt adeep supervision strategy for each sub-side output prediction map from the decoder and calculatethe binary cross-entropy (BCE) [40] loss.
The total loss L D for the decoder of the proposed networkcould be expressed as follows:
L D = (cid:80) Ii L ( i ) d , where L ( i ) d represents the loss of the i -th sub-sideoutput prediction.
For the other two aided branches in FGA module, we also use BCE loss to obtainthe losses L Con and L LR .
Unlike the decoder, we do not use a deep supervision strategy.
The ﬁnalloss for the entire network is deﬁned as follows:
L total = a L D + b L Con + c L LR , where a, b , and care the hyper-parameters to balance different loss terms.
In this paper, we set a = 1 , b = 0 .
9 , andc = 0 .
9 , respectively.
Note that all experiments use the same hyper-parameters.
Experiments
[Figure]
To investigate the validity of our proposed FigureGround Aided module (FGA), we design and establish a set of tasks to evaluate the Figure-Ground assignment ability of deep convolutional neural networks, inspired by the Figure-Ground Segregationtest in cognitive science experiments [26, 46, 49, 55]as shown in Fig.
4 (a).
This task is described in detailas shown in Fig.
4 (b), by sampling the content ofa randomly given texture image and ﬁlling the foreground and background of the given image using that
UNet UNet+Edge UNet+FGA 0.
DeepLabv3 DeepLabv3+Edge DeepLab+FGA 0.79 0.8 0.
Baseline +Edge +FGA
Hard
Hard
Easy Easy
Normal Normal
Easy Easy %
[Figure]
training data
I o U [Figure]
Baseline
[Figure]
texture content to generate the desired sample ﬁnally.
Here we use a synthetic dataset to test a deepconvolutional neural network to remove distracting factors.
Standard computer vision datasets makeit difﬁcult to pinpoint the relative contributions of different visual strategies since the performance ofarchitecture may be affected by several factors, including dataset biases, model hyper-parameters,and the number of samples [31, 22].
Test Design.
We use segmentation labels from the Pascal VOC [8] dataset and the rich texturedataset (DTD [7]) to synthesize our dataset.
Referring to the cognitive science experiments whichexclude irrelevant factors from the test conditions, such as excluding other irrelevant factors for
Figure-Ground Segregation, we also need to exclude the inﬂuence of several factors, i.e. , 1) excludingthe inﬂuence of contextual information brought by semantic labels since utilizing semantic labelsof different objects helps to learn discriminative feature representation; 2) excluding the inﬂuenceof spatial contexts provided by multiple instances belonging to the same class since they contributeto learning robust ( e.g. , scale-invariant or occlusion-aware) feature representation; and 3) excludingsegmentation cues due to signiﬁcant appearance differences between foreground and backgroundsince their distinct textures, colors, and illumination help to learn cheap features to distinguish them.
Guided by the above principles, the whole process of sample generation can be divided into thefollowing steps: 1) select a random image in the Pascal dataset and use one of the object instances asa ﬁgure and the remaining regions as the ground; 2) given a collection of texture images, randomlyselect a texture image from it; 3) two random transformations (such as rotation and scaling) areperformed on the texture image independently; and 4) ﬁll the ﬁgure region and the ground regionwith the two transformed textures respectively, resulting in a synthetic sample.
In addition, for a morecomprehensive evaluation, three datasets of different difﬁculty levels (Easy, Normal, and Hard) areestablished by varying the transformations, the division of the set of texture images, and the size ofﬁgure regions.
Each dataset contains 2,500 unique images with a 224 × 224 resolution, split intotraining (2,000) and test (500) sets.
Experimental Conditions.
In this paper, we conduct experiment on two representative architectures:
U-Net [50] and Deeplabv3 [6].
U-Net is one of the most popular architectures in the ﬁeld ofsegmentation, which has a typical encoder-decoder architecture, and the shallow features are directlyfed into the decoder through skip connections.
Deeplabv3 is a highly competitive network forsemantic segmentation.
Through atrous spatial pyramid pooling, it can expand the receptive ﬁeldwhile ensuring detailed modeling.
Each model is trained using the Adam optimizer with a batch sizeof 16 and a learning rate of 1 e − 4 for the Figure-Ground Segregation test.
To isolate the inﬂuence ofpre-trained weights, all weights are randomly initialized.
We train each model for 20,000 iterations.
For the evaluation of the results, we focus on the differences between the output and the groundtruth in terms of structure and ﬁlling completeness, so we choose S-measure [9], and IoU [33] asevaluation metrics. [Figure]
Covx
[Figure]
[Table]
Result and Discussion.
Through experimentson the Figure-Ground segregation test, we canget the following four conclusions.
(cid:73) First , withdifferent difﬁculty levels, the proposed FGAcan consistently improve the performance ofthe base model (in Fig.
5 (a)).
It shows thatour method has a strong capability of the FigureGround assignment by introducing human cognitive cues.
(cid:73) Second , as shown in Fig.
5 (a),our proposed FGA-Net has a better performancecompared to the straightforward strategy of directly adding edge supervision.
Its segmentationresults are more complete in terms of both structure and ﬁlling of foreground regions as shown in Fig.5 (c).
Using the edge supervision can only provide local perception at the foreground boundary, whilemore structural priors and perception of the foreground region are required for the Figure-Groundassignment.
(cid:73) Third , as shown in Fig.
5 (b), it can be seen that as the number of training samplesdecreases, our model still outperforms the contenders.
Our FGA trained on only 1% of the datasetis comparable to Baseline+Edge trained on 10% of the dataset, suggesting that FGA is more dataefﬁcient.
The supervisions from the Figure-Ground cues help the network learn an inductive biasthat the network focuses on the foreground and background and their differences at the boundaries,thereby reducing the amount of data required.
(cid:73) Fourth , as shown in Fig.
6, the LR cue provides amore complete edge for segmentation results, the Convexity cue enhances the perception of shapedetail, and the IEM enhances the consistency within the foreground.
In addition, we ﬁnd that usingtwo branches to learn both priors separately is more effective than using a single branch to learn bothpriors.
This phenomenon explains to some extent the certain orthogonality between the two cues,which depend on different features.
And we also conduct an ablation study on the IEM, as shown in
Table 1, which identiﬁes the performance contribution of each sub-module in the IEM.
It is worthmentioning that the local interactions bring more improvement than the global interactions.
This maybe due to the boundary assignment process that tends to capture discriminative features from the localregion.
3.2 Applications
We verify the effectiveness of our proposed method on four challenging visual tasks, i.e. , Camouﬂaged
Object Detection ( COD ) [11], Polyp Segmentation ( PS ) [12], and Lung Infection Segmentation ( LIS )[13].
The COD task mainly manifests in complex background/edges, small targets/structures, slendertrunks/limbs, and partially occluded objects.
The main feature of the PS task is its high diversity ofsize and texture.
The main difﬁculty of the LIS task lies in the complex structures of the foreground.
In addition, we also verify the effectiveness of our proposed method on the non-challenging ﬁgureground task.
Camouﬂaged Object Detection.
Datasets: We evaluate our model on three challenging visual
Camouﬂaged Object Detection datasets, i.e. , CHAMELEON (CHA) [59], CAMO (CAM) [27], and
COD10K (COD) [11].
CHA is collected from the Internet by searching “camouﬂaged animal” as thekeyword.
CAM is the ﬁrst publicly released COD dataset.
COD is the most challenging dataset withhigher image quality.
Implementation Details: We implement our model with PyTorch, and TITAN
Xp GPUs are used for training and testing.
ResNet-50 [17] is used as a encoder in our model, which isinitialized as the pre-trained weights on ImageNet.
In this paper, we use the resolution of 480 × [-
 [Figure] [Figure] [Figure]-]480during both the training phase and testing phase.
Our model is trained for 100 epochs using the Adam
[Table]
[Table] [Table]
[Table]
ET I S [ 58 ]
[Table]
[23] optimizer with an initial learning rate of 0.0001, decreased by 0.1 at 50 epochs.
The batch size is32.
Evaluation Criteria: S-measure ( S α ) [9], mean E-measure ( E φ ) [37], weighted F-measure ( F ωβ )[37], and
Mean Absolute Error ( MAE ) [42] are used as the evaluation metrics.
Comparison with
SOTA: We compare the performance of FGA-Net with SOTA methods, including EGNet [72], SINet[11], SINetv2 [10], MGL [69], PFNet [38], and UGTR [68].
Quantitative results are listed in Table2.
Obviously, our model outperforms the contenders.
In particular, FGA-Net achieves signiﬁcantperformance improvement compared to the second-best method SINetv2 [10] on the challenging
COD10K dataset.
It proves that with the help of Figure-Ground assignment mechanism, it can makethe network more efﬁcient in solving the problem of separating ambiguous regions, which is causedby camouﬂaged objects that are highly similar to background in appearance.
Polyp Segmentation.
Datasets: We evaluate our model on ﬁve challenging Polyp Segmentation( PS ) datasets, Kvasir (Kvasir)
[18], CVC-ClinicDB/CVC-612 (CVC-612) [2], CVC-ColonDB(ColonDB)
[60], ETIS (ETIS) [58], and EndoScene (Endo) [61].
Kvasir is the most challengingdataset, which contains 1,000 images.
CVC-612 dataset includes 612 open-access images from31 colonoscopy clips.
ColonDB and ETIS consist of 380 and 196 polyp images.
Note that thetraining set only contains the data of Kvasir and CVC-612.
We follow [12] to split all datasets.
Implementation Details: We use the same implementation method as COD task, except that thebatch size is 16.
We use the resolution of 352 × 352 during both the training phase and testing phase.
Evaluation Criteria: Dice [19], IoU [18], F ωβ , S α , maximum E-measure ( E maxφ ), and MAE areused as the metrics.
Comparison with SOTA: We compare our FGA-Net with four SOTA medicalimage segmentation methods, including UNet [50], UNet++ [76], SFA [14], Pra [12], and MSNet[74].
Quantitative results are listed in Table 5.
We can observe that our FGA-Net surpasses the aforementioned methods in most of the metrics.
Compared to the second-best methods on eachdataset, FGA-Net outperforms all of them on ﬁve datasets.
Similar results can also be observed interms of the MAE score.
Lung Infection Segmentation.
Datasets: We also evaluate our model for Lung Infection Segmentation ( LIS ) on the COVID-
CT segmentation dataset, including 100 axial CT images from different
COVID-19 patients.
We follow [13] to randomly select 45 CT images as training samples, 5 imagesfor validation, and 50 images for testing.
Implementation details: We use the same implementationmethod as COD task, except that the batch size is 24.
We use the resolution of 352 × 352 during boththe training phase and testing phase.
Evaluation Criteria: Dice, Sensitivity (Sen.) [56], Speciﬁcity(Spec.)
[57], S α , E meanφ , and MAE are used as the metrics.
Comparison with SOTA: We compareour FGA-Net with six SOTA medical image segmentation methods including UNet [50], UN++ [76],
AUN [41], GUN [54], DUN [29], Inf [13], and ERRNet [20].
Quantitative results are listed in Table3.
We can observe that our model consistently outperforms all other contenders across all metrics.
Non-Challenging Figure-Ground Task.
In Table 4, we conduct experiments on more nonchallenging Figure-Ground datasets (salient object detection), and the results show that the FGA-Netproposed in our paper is also competent for salient object detection.
It indicates that FGA helps fordifferent segmentation tasks by improving the boundary assignment capability of the model, which inturn improves the ﬁnal performance.
Related Work
Historically, the visual phenomenon most closely associated with the perceptual organization isgrouping, and Figure-Ground assignment [24, 66, 53, 25].
In general, grouping determines what thequalitative elements of perception are, and ﬁgure-ground assignment determines the interpretationof those elements in terms of their shapes and relative locations in the layout of surfaces in thereal-world [63, 64].
Figure-Ground assignment refers to the perceptual process of assigning aboundary separating two regions to one of them [52] and provides important conﬁgural prior forscene perception.
Accordingly, the studies of the Figure-Ground assignment process [43, 63, 64]show that it plays a central role in aiding higher levels of visual perception.
Therefore, we hope to introduce the Figure-Ground assignment process into the segmentation modelto help address the object segmentation.
Factors that affect Figure-Ground assignment includesize, surroundedness, orientation and contrast [52], symmetry [15], parallelism [39], convexity[39, 21, 44, 34], meaningfulness [43], and lower region [62].
In this paper, we mainly consider thecues related to the conﬁguration, i.e. , convexity and lower region.
Compared to recent research[35, 36] applying the Figure-Ground Assignment mechanism in some vision tasks without consideringhigh-level supervisions, our work instead explores an end-to-end ﬁgure-ground aided approach thatcan be easily incorporated into existing encoder-decoder models and improve their performance.
Conclusion
In this paper, we demonstrate that the Figure-Ground cues inspired by the perceptual organization ofhuman vision can be effectively utilized to improve the performance of a CNN model for differentchallenging robust perception organization tasks, e.g. , camouﬂaged object detection, polyp segmentation, and lung infection segmentation.
Speciﬁcally, we investigate the proposed FGA module, on acarefully established synthetic dataset for the Figure-Ground Segregation test, which aims to measurethe capability of Figure-Ground Assignment.
The empirical study shows that the proposed newmodule consistently improves the performance of several representative networks in reducing visualambiguities.
Moreover, the model implemented via our proposed FGA mechanism outperforms SOTAapproaches in three challenging real-world applications.
We believe this study provides valuableinsights to the community and attracts attention to exploring the Figure-Ground principles for othervisual perception tasks.
Societal Impact.
Perceptual organization is one of the most challenging computer vision tasks.
Albeitbeing challenging, Perceptual organization is beneﬁcial to a wide range of applications.
However,there also exists the risk that the technology is utilized in the scenario of the illegal shoot, maliciousedit, and incorrect use.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes] See Abstract and Section 1.
(b) Did you describe the limitations of your work?
[Yes] See Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See
Section 5.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See supplemen-
tal material.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 3.1 and Section 3.2.
(c) Did you report error bars (e g., with respect to the random seed after running experi-
ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 3.2.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section 3.1 and
Section 3.
(b) Did you mention the license of the assets?
[No] All of the datasets we used are publicly
available for research.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]