Exploring Figure-Ground Assignment Mechanism in Perceptual Organization
Abstract
Perceptual organization is a challenging visual task that aims to perceive and group the individual visual element so that it is easy to understand the meaning of the scene as a whole.
Most recent methods building upon advanced Convolutional Neural Network (CNN) come from learning discriminative representation and modeling context hierarchically.
However, when the visual appearance difference between foreground and background is obscure, the performance of existing methods degrades signiﬁcantly due to the visual ambiguity in the discrimination process.
In this paper, we argue that the ﬁgure-ground assignment mechanism, which conforms to human vision cognitive theory, can be explored to empower CNN to achieve a robust perceptual organization despite visual ambiguity.
Speciﬁcally, we present a novel Figure-Ground-Aided (FGA) module to learn the conﬁgural statistics of the visual scene and leverage it for the reduction of visual ambiguity.
Particularly, we demonstrate the beneﬁt of using stronger supervisory signals by teaching (FGA) module to perceive conﬁgural cues, i.e. , convexity and lower region, that human deem important for the perceptual organization.
Furthermore, an Interactive Enhancement Module (IEM) is devised to leverage such conﬁgural priors to assist representation learning, thereby achieving robust perception organization with complex visual ambiguities.
In addition, a well-founded visual segregation test is designed to validate the capability of the proposed FGA mechanism explicitly.
Comprehensive evaluation results demonstrate our proposed FGA mechanism can effectively enhance the capability of perception organization on various baseline models.
Nevertheless, the model augmented via our proposed FGA mechanism also outperforms state-of-the-art approaches on four challenging real-world applications.
Introduction
Perceptual organization, which is a vital visual task, refers to the processes by which the disjoint bits of visual information are structured into the larger coherent units that we eventually experience as environmental objects.
Thanks to the developments of discriminative representation learning and hierarchical context modeling with convolutional neural networks [17, 33, 48, 77], the past few years have witnessed tremendous progress in perceptual organization.
Recent advanced methods are usually based on the framework of fully convolutional network (FCN) [5, 33, 50, 76], which learn a discriminative feature representation and hierarchically model the local context by supervision derived from human-given labels.
However, the performance of existing methods degrades signiﬁcantly when deployed in some challenging tasks with complex visual ambiguity, such as camouﬂaged object detection [11, 27], medical image segmentation [12, 50], and visual industrial detection [74].
Since the visual appearance differences between the foreground and background are obscure, it is difﬁcult to perceive the correlation between individual visual elements and determine the boundaries.
The visual ambiguities impede CNN’s represent learning and contextual modeling, leading to inaccurate and incomplete perceptual organization.
[Figure]
[Figure]
To address this issue, more perceptual knowledge is required to be incorporated into visual organization.
In the 1920 s, the Gestalt psychologists identiﬁed Grouping and Figure-Ground as two important principles underlying the process of perceptual organization [24, 25, 53, 66].
Grouping principle refers to bring together individual visual elements that produce stimuli to form a holistic perception.
It has attracted more attention because it is intuitively similar to the contextual modeling ability of CNN.
Different from Grouping, Figure-Ground assignment refers to the perception that assigns a boundary separating two regions to one of them [52].
By introducing the conﬁgural statistics of the natural world in which the visual system evolved [3], the Figure-Ground assignment principle is thought to be important in reducing the visual ambiguousness of a scene.
In addition, Cognitive science studies [15, 47, 81] have found that neural activity associated with the Figure-Ground assignment mechanism in the V 2 cortex of human vision occurs as early as 10 - 25 ms after the generation of visual stimuli, providing strong support for the role of local bottom-up processing.
Another study [43] on the “meaningfulness principle” also showed that assigned ﬁgures tend to be associated with neighborhoods with familiar shapes, pointing to the integration of knowledge from the top-down.
Inspired by these studies, as shown in Fig.
1, we argue that exploring ﬁgure-ground assignment mechanism can empower CNN the ability of perceptual organization despite visual ambiguity, and consequently presents a novel Figure-Ground-Aided (FGA) module that learns the conﬁgural statistics of visual scene, and leverages it for representation learning.
Speciﬁcally, we ﬁrstly investigate the conﬁgural cues related to the Figure-Ground assignment mechanism in human psychophysics and ﬁnd that the ﬁgural region usually takes on the shape instructed by the separating boundary and appears closer to the viewer, while the ground region is seen as extending behind the ﬁgure [43].
Typically, as shown in Fig.
2 (a), Convexity cue [21, 34, 39, 44] corresponds to the regions on either side of the boundary where the scene depth changes abruptly and is beneﬁcial for analyzing the hierarchical relationship between neighboring regions in the image, facilitating hierarchical contextual modeling.
And lower region cues [16, 62] usually corresponds to the region of the scene where occlusion has occurred and is beneﬁcial for analyzing the occlusion relationship between various neighborhoods in an image and determining the shape attribution of foreground objects and background regions.
After that, instead of using a weak form of directly using mask labels as supervision, we refer to the method in [32] of teaching the FGA module to perceive the conﬁgural cues that human deem important for the perceptual organization by using human-derived labels as stronger supervisory signals.
Notably, the Ground-Truth (GT) segmentation masks imply the prior knowledge of the annotators’ understanding of natural scenes.
Therefore, we easily generate the labels of the two conﬁgural cues from the Ground-Truth (GT) label without the intervention of additional information and exploit them as supervisions for the FGA module, facilitating the modeling of the conﬁgural statistics of natural scenes.
Furthermore, an Interactive Enhancement Module (IEM) is presented to progressively enhance the discriminative features for boundary assignment via a local/global interactive strategy.
Speciﬁcally, a collaborative local interaction process is ﬁrst introduced by swapping queries to align locally between contextual co-occurring conﬁgural features.
Then, a global interaction process is introduced to establish global spatial correlations of local conﬁgural features, resulting in complete object boundaries.
Moreover, a Lambda strategy is utilized to improve the computational efﬁciency and performance of the original self-attention module.
F
G G
F
G
F
L o w er
R e g i o n
C o n v e x i t y
F
G #1 # # # # # # #
GT
LR
Convexity
[Figure]
To systematically investigate the performance of our proposed method for the
Figure Ground assignment, we design a synthetic computer vision task inspired by an important experiment in cognitive science—Figure-Ground Segregation [26, 46, 49, 55].
Furthermore, the comprehensive experiments demonstrate that our proposed FGA module can facilitate the CNN to learn more efﬁciently in the reduction of visual ambiguities with low data requirements.
Finally, we also validate the performance of our proposed mechanism in four challenging visual applications, including camouﬂaged object detection [11, 71], polyp detection [12], and lung anomaly detection [13].
The results demonstrate the superiority of our method over SOTA methods.
The contributions are summarized as follows:
(1) This paper explores the Figure-Ground assignment mechanism
from human vision cognitive theory to empower CNN to learn conﬁgural statistics to reduce visual ambiguities, thereby achieving robust perceptual organization.
This paper presents a novel Figure-Ground-Aided module to integrate ﬁgure-ground-aided cues in a hierarchical manner.
An Interactive Enhancement Module is devised to progressively enhance the discriminative features for Figure-Ground assignment via a local/global interactive strategy.
(3) This paper introduces the Figure-Ground Segregation test, a synthetic visual perception challenge to systematically investigate the performance of models in Figure-Ground assignment.
Experimental results demonstrate our proposed FGA module consistently improves the Figure-Ground assignment performance of several representative networks on datasets of different difﬁculty levels.
(4) Extensive experiments are also performed on four challenging robust object segmentation applications, showing that the model constructed via our FGA module outperforms SOTAs.
Methodology
Architecture.
In this paper, we propose a novel Figure-Ground-Aided (FGA) module that uses conﬁguration cues from the Figure-Ground assignment process to enhance the perception of foregroundbackground relationship, thereby achieving a robust perceptual organization result.
It can be easily incorporated into existing encoder-decoder models and improve their performance.
Fig.
3 shows the overall architecture of the proposed Figure-Ground-Aided module.
Our proposed FGA module is composed of two cue-aided branches and an Interactive Enhancement Module (IEM).
Here, we use ResNet-50 [17] as the backbone.
Speciﬁcally, we remove the fully connected layer and retain all convolutional bocks.
Given an input image of shape H × W , this backbone will generate ﬁve scales of features with spatial resolution gradually decreasing by stride 2.
We denote these features as E =
{ E k | k
= 1 , 2 , 3 , 4 , 5 } .
The size of the k -th feature is (cid:2) C k , H 2 k , W 2 k (cid:3) , where C k is the number of channels of the k -th feature.
We utilize the feature from { E k | k
= 1 , 2 , 3 , 4 } as skip-connected feature for FGA module and the decoder. D = { D k | k = 1 , 2 , 3 , 4 , 5 } are the feature maps of the decoder part.
The intermediate features E Conk and E LRk are derived from the feature E 5 .
The calculation process here is the same as in the Decoder part.
E ncode r
Decoder E
E
E E
E
D 2 D 3 D 4 D
D
Supervision
Side Output
Skip Connection
C Concatenation
Top-Down information Flow
Bottom-Up information Flow
M
E Con
E LR
E
Lambda
Q Q
Lambda Lambda
Collaboration
Y
E G V
K
V K E LR 1 E LR
E Con 1 E Con
E LR 2 E LR
E Con 2 E Con
E Con 3 E Con
E Con 4 E Con
E LR 3 E LR 3 E LR 4 E LR
Cue L Cue L
Cue C
[Figure]
C
Loss
Input
GT
Generate Figure-Ground Cues
U p s a m p l e C
Aggregation Module U p s a m p l e C
Aggregation Module
G l oba l I n t e r ac t i on
G l oba l I n t e r ac t i on
Interactive Enhancement Module IEM Interactive Enhancement Module IEM Interactive Enhancement Module IEM
C o ll abo r a t
i v e
Loca l
I n t e r ac t i on
C o ll abo r a t
i v e
Loca l
I n t e r ac t i on
M
We leverage the convexity and lower region cues, which can be calculated directly from the GT label and do not require any additional information, as supervisions for perceiving the contextual information of object boundary, respectively.
We ﬁrst allow the network to learn the statistical features related to the conﬁguration in a top-down manner.
Then we use the learned features to guide the bottom-up features from the encoder to integrate bottom-up and top-down information.
Interactive
Enhancement Module (IEM) , which consists of two parts: Collaborative Local Interaction (CLI) and Global Interaction (GI).
IEM has three input features ( E Con , E LR , and E ), two of which come from the intermediate feature learned by the Figure-Ground assignment cues aided branches ( E Con and E LR ).
“Lambda” indicates the use of a Lambda strategy [1] to improve the computation efﬁciency of the CLI and GI interaction.
assignment commonly thought to region segmentation, and it is an step in forming perception of surfaces, shapes, and objects that when humans observe images, they will use some conﬁgural cues to distinguish between foreground and background, including Convexity [21, 34, 39, 44]
Lower
Accordingly, we propose to introduce the convexity and lower region cues, as shown in Fig.
2 (b).
To generate the convexity cue, we used morphological opening and closing operations [4, 45].
It can be formed as: Cue C =
Cat ( Cue C 1 , Cue C 2 )
, “ Cat ” is the concatenation operation.
[Equation]
where GT is the ground truth segmentation mask.
Φ ◦ ( . ) is the opening operation.
Φ • ( . ) is the closing operation.
K C is the structure element whose size is 10.
We get the convexity Cue by concatenating
Cue C 1 and Cue C 2 , whose shape is [2 , H, W ] .
H and W are the height and width of the mask.
To generate the lower region cue, we used morphological erosion and expansion operations.
It can be expressed as: Cue L =
[Equation]
[Equation]
[Equation]
(4) where Φ (cid:9) ( . ) is an erosion operation.
Φ ⊕ ( . ) is the expansion operation.
K L is the structure element whose size is 5.
We get lower region cue by concatenating Cue L 1 and Cue L 2 .
Interactive Enhancement Module.
To effectively utilize the important conﬁgural cues (convexity and lower region) provided by the Figure-Ground assignment mechanism, a novel Interactive Enhancement Module (IEM) is devised.
The IEM has three input features, two of which come from the intermediate feature learned by the Figure-Ground assignment cue-aided branches.
We deﬁne them as E Con and E LR , respectively.
Moreover, the remaining branch is the skip-connection feature E from the encoder.
We further improve the representation ability of the encoder feature by establishing the interactions among the features of the skip-connection features and aided branches feature.
As shown in Fig.
3, the interactions in the IEM consist of collaborative local interaction and global interaction.
1) Collaborative Local (CLI) interaction: the local interaction of the two Figure-Ground cues integrates the local structural context features.
Global interaction (GI): the global interaction between base features and local contextual features aims to enhance foreground features and suppress the background with structural context features.
The detailed structure of the Interactive Enhancement Module is illustrated in Fig.
We use a spatial self-attention mechanism that considers both content and location interactions to achieve local or global interaction of features.
This paper uses an efﬁcient Lambda layer [1], which captures interactions of feature elements by transforming available contexts into linear functions, termed lambdas, and applying these linear functions to each input separately.
For the collaborative local interaction part, it can be expressed as follows:
[Equation]
where Y is the output of CLI.
P denotes the relative position embeddings.
Q LR is the query, which is computed by: Q LR = E LR W LR Q .
Q Con is calculated in the same way.
W LR Q is the learnable weights.
Lambda ( . ) is deﬁned as:
[Equation]
where the range of relative position in CLI is set to 5.
Cue ∈ LR, Con
For the global interaction part, it can be expressed as follows:
[Equation]
where E G =
Cat ( Y, E ) .
E G and M correspond to the feature maps shown on the right side of the
IEM in Fig.
The Lambda ( . ) function is deﬁned as follows:
[Equation]
where the range of relative position in GI is set to the whole feature map size.
Loss Function.
In this section, we introduce the loss function used to train FGA-Net.
We adopt a deep supervision strategy for each sub-side output prediction map from the decoder and calculate the binary cross-entropy (BCE) [40] loss.
The total loss L D for the decoder of the proposed network could be expressed as follows:
L D = (cid:80) Ii L ( i ) d , where L ( i ) d represents the loss of the i -th sub-side output prediction.
For the other two aided branches in FGA module, we also use BCE loss to obtain the losses L Con and L LR .
Unlike the decoder, we do not use a deep supervision strategy.
The ﬁnal loss for the entire network is deﬁned as follows:
L total = a L D + b L Con + c L LR , where a, b , and c are the hyper-parameters to balance different loss terms.
In this paper, we set a = 1 , b = 0 .
9 , and c = 0 .
9 , respectively.
Note that all experiments use the same hyper-parameters.
Experiments 3.1 Figure-Ground Segregation Test
[Figure]
To investigate the validity of our proposed FigureGround Aided module (FGA), we design and establish a set of tasks to evaluate the Figure-Ground assignment ability of deep convolutional neural networks, inspired by the Figure-Ground Segregation test in cognitive science experiments [26, 46, 49, 55] as shown in Fig.
4 (a).
This task is described in detail as shown in Fig.
4 (b), by sampling the content of a randomly given texture image and ﬁlling the foreground and background of the given image using that texture content to generate the desired sample ﬁnally.
Here we use a synthetic dataset to test a deep convolutional neural network to remove distracting
UNet UNet+Edge UNet+FGA 0.
DeepLabv3 DeepLabv3+Edge DeepLab+FGA 0.79 0.
[Figure]
Hard
Hard
Easy Easy
[Figure]
Baseline
[Figure]
factors.
Standard computer vision datasets make it difﬁcult to pinpoint the relative contributions of different visual strategies since the performance of architecture may be affected by several factors, including dataset biases, model hyper-parameters and the number of samples [22, 31].
Test Design.
We use segmentation labels from the Pascal VOC [8] dataset and the rich texture dataset (DTD [7]) to synthesize our dataset.
Referring to the cognitive science experiments which exclude irrelevant factors from the test conditions, such as excluding other irrelevant factors for
Figure-Ground Segregation, we also need to exclude the inﬂuence of several factors, i.e. , 1) excluding the inﬂuence of contextual information brought by semantic labels since utilizing semantic labels of different objects helps to learn discriminative feature representation; 2) excluding the inﬂuence of spatial contexts provided by multiple instances belonging to the same class since they contribute to learning robust ( e.g. , scale-invariant or occlusion-aware) feature representation; and 3) excluding segmentation cues due to signiﬁcant appearance differences between foreground and background since their distinct textures [72, 73], colors, and illumination help to learn cheap features to distinguish them.
Guided by the above principles, the whole process of sample generation can be divided into the following steps: 1) select a random image in the Pascal dataset and use one of the object instances as a ﬁgure and the remaining regions as the ground; 2) given a collection of texture images, randomly select a texture image from it; 3) two random transformations (such as rotation and scaling) are performed on the texture image independently; and 4) ﬁll the ﬁgure region and the ground region with the two transformed textures respectively, resulting in a synthetic sample.
In addition, for a more comprehensive evaluation, three datasets of different difﬁculty levels (Easy, Normal, and Hard) are established by varying the transformations, the division of the set of texture images, and the size of ﬁgure regions.
Each dataset contains 2,500 unique images with a 224 × 224 resolution, split into training (2,000) and test (500) sets.
Experimental Conditions.
In this paper, we conduct experiment on two representative architectures:
U-Net [50] and Deeplabv3 [6].
U-Net is one of the most popular architectures in the ﬁeld of segmentation, which has a typical encoder-decoder architecture, and the shallow features are directly fed into the decoder through skip connections.
Deeplabv3 is a highly competitive network for semantic segmentation.
Through atrous spatial pyramid pooling, it can expand the receptive ﬁeld while ensuring detailed modeling.
Each model is trained using the Adam optimizer with a batch size of 16 and a learning rate of 1 e − 4 for the Figure-Ground Segregation test.
To isolate the inﬂuence of pre-trained weights, all weights are randomly initialized.
We train each model for 20,000 iterations.
For the evaluation of the results, we focus on the differences between the output and the ground truth in terms of structure and ﬁlling completeness, so we choose S-measure [9], and IoU [33] as evaluation metrics.
[Figure]
[Table]
Result and Discussion.
Through experiments on the Figure-Ground segregation test [Table] we can get the following four conclusions.
(cid:73) First , with different difﬁculty levels, the proposed FGA can consistently improve the performance of the base model (in Fig.
5 (a)).
It shows that our method has a strong capability of the FigureGround assignment by introducing human cognitive cues.
(cid:73) Second , as shown in Fig.
5 (a), our proposed FGA-Net has a better performance compared to the straightforward strategy of directly adding edge supervision.
Its segmentation results are more complete in terms of both structure and ﬁlling of foreground regions as shown in Fig. 5 (c).
Using the edge supervision can only provide local perception at the foreground boundary, while more structural priors and perception of the foreground region are required for the Figure-Ground assignment.
(cid:73) Third , as shown in Fig.
5 (b), it can be seen that as the number of training samples decreases, our model still outperforms the contenders.
Our FGA trained on only 1% of the dataset is comparable to Baseline+Edge trained on 10% of the dataset, suggesting that FGA is more dataefﬁcient.
The supervisions from the Figure-Ground cues help the network learn an inductive bias [68] that the network focuses on the foreground and background and their differences at the boundaries, thereby reducing the amount of data required.
(cid:73) Fourth , as shown in Fig.
6, the LR cue provides a more complete edge for segmentation results, the Convexity cue enhances the perception of shape detail, and the IEM enhances the consistency within the foreground.
In addition, we ﬁnd that using two branches to learn both priors separately is more effective than using a single branch to learn both priors.
This phenomenon explains to some extent the certain orthogonality between the two cues, which depend on different features.
And we also conduct an ablation study on the IEM, as shown in
Table 1, which identiﬁes the performance contribution of each sub-module in the IEM.
It is worth mentioning that the local interactions bring more improvement than the global interactions.
This may be due to the boundary assignment process that tends to capture discriminative features from the local region.
3.2 Applications
We verify the effectiveness of our proposed method on four challenging visual tasks, i.e. , Camouﬂaged
Object Detection ( COD ) [11], Polyp Segmentation ( PS ) [12], and Lung Infection Segmentation ( LIS ) [13].
The COD task mainly manifests in complex background/edges, small targets/structures, slender trunks/limbs, and partially occluded objects.
The main feature of the PS task is its high diversity of size and texture.
The main difﬁculty of the LIS task lies in the complex structures of the foreground.
In addition, we also verify the effectiveness of our proposed method on the non-challenging ﬁgureground task.
Camouﬂaged Object Detection.
Datasets: We evaluate our model on three challenging visual
Camouﬂaged Object Detection datasets, i.e. , CHAMELEON (CHA) [59], CAMO (CAM) [27], and
COD10K (COD) [11].
CHA is collected from the Internet by searching “camouﬂaged animal” as the keyword.
CAM is the ﬁrst publicly released COD dataset.
COD is the most challenging dataset with higher image quality.
Implementation Details: We implement our model with PyTorch, and TITAN
Xp GPUs are used for training and testing.
ResNet-50 [17] is used as a encoder in our model, which is initialized as the pre-trained weights on ImageNet.
In this paper, we use the resolution of 480 ×during both the training phase and testing phase.
Our model is trained for 100 epochs using the Adam
[Table] [Table]
[Table] [Table]
ET I S [ 58 ]
[Table]
[23] optimizer with an initial learning rate of 0.0001, decreased by 0.1 at 50 epochs.
The batch size is 32.
Evaluation Criteria: S-measure ( S α ) [9], mean E-measure ( E φ ) [37], weighted F-measure ( F ωβ ) [37], and
Mean Absolute Error ( MAE ) [42] are used as the evaluation metrics.
Comparison with
SOTA: We compare the performance of FGA-Net with SOTA methods, including EGNet [78], SINet [11], SINetv2 [10], MGL [70], PFNet [38], and UGTR [69].
Quantitative results are listed in Table 2.
Obviously, our model outperforms the contenders.
In particular, FGA-Net achieves signiﬁcant performance improvement compared to the second-best method SINetv2 [10] on the challenging
COD10K dataset.
It proves that with the help of Figure-Ground assignment mechanism, it can make the network more efﬁcient in solving the problem of separating ambiguous regions, which is caused by camouﬂaged objects that are highly similar to background in appearance.
Polyp Segmentation.
Datasets: We evaluate our model on ﬁve challenging Polyp Segmentation ( PS ) datasets, Kvasir (Kvasir)
[18], CVC-ClinicDB/CVC-612 (CVC-612) [2], CVC-ColonDB (ColonDB) [60], ETIS (ETIS) [58], and EndoScene (Endo) [61].
Kvasir is the most challenging dataset, which contains 1,000 images.
CVC-612 dataset includes 612 open-access images from 31 colonoscopy clips.
ColonDB and ETIS consist of 380 and 196 polyp images.
Note that the training set only contains the data of Kvasir and CVC-612.
We follow [12] to split all datasets.
Implementation Details: We use the same implementation method as COD task, except that the batch size is 16.
We use the resolution of 352 × 352 during both the training phase and testing phase.
Evaluation Criteria: Dice [19], IoU [18], F ωβ , S α , maximum E-measure ( E maxφ ), and MAE are used as the metrics.
Comparison with SOTA: We compare our FGA-Net with four SOTA medical image segmentation methods, including UNet [50], UNet++ [82], SFA [14], Pra [12], and MSNet
Quantitative results are listed in Table 5.
We can observe that our FGA-Net surpasses the aforementioned methods in most of the metrics.
Compared to the second-best methods on each dataset, FGA-Net outperforms all of them on ﬁve datasets.
Similar results can also be observed in terms of the MAE score.
Lung Infection Segmentation.
Datasets: We also evaluate our model for Lung Infection Segmentation ( LIS ) on the COVID-
CT segmentation dataset, including 100 axial CT images from different
COVID-19 patients.
We follow [13] to randomly select 45 CT images as training samples, 5 images for validation, and 50 images for testing.
Implementation details: We use the same implementation method as COD task, except that the batch size is 24.
We use the resolution of 352 × 352 during both the training phase and testing phase.
Evaluation Criteria: Dice, Sensitivity (Sen.) [56], Speciﬁcity (Spec.)
[57], S α , E meanφ , and MAE are used as the metrics.
Comparison with SOTA: We compare our FGA-Net with six SOTA medical image segmentation methods including UNet [50], UN++ [82],
AUN [41], GUN [54], DUN [29], Inf [13], and ERRNet [20].
Quantitative results are listed in Table 3.
We can observe that our model consistently outperforms all other contenders across all metrics.
Non-Challenging Figure-Ground Task.
In Table 4, we conduct experiments on more nonchallenging Figure-Ground datasets (salient object detection), and the results show that the FGA-Net proposed in our paper is also competent for salient object detection.
It indicates that FGA helps for different segmentation tasks by improving the boundary assignment capability of the model, which in turn improves the ﬁnal performance.
Related Work
Historically, the visual phenomenon most closely associated with the perceptual organization is grouping, and Figure-Ground assignment [24, 25, 53, 66].
In general, grouping determines what the qualitative elements of perception are, and ﬁgure-ground assignment determines the interpretation of those elements in terms of their shapes and relative locations in the layout of surfaces in the real-world [63, 64].
Figure-Ground assignment refers to the perceptual process of assigning a boundary separating two regions to one of them [52] and provides important conﬁgural prior for scene perception.
Accordingly, the studies of the Figure-Ground assignment process [43, 63, 64] show that it plays a central role in aiding higher levels of visual perception.
Therefore, we hope to introduce the Figure-Ground assignment process into the segmentation model to help address the object segmentation.
Factors that affect Figure-Ground assignment include size, surroundedness, orientation and contrast [52], symmetry [15], parallelism [39], convexity [21, 34, 39, 44], meaningfulness [43], and lower region [62].
In this paper, we mainly consider the cues related to the conﬁguration, i.e. , convexity and lower region.
Compared to recent research [35, 36] applying the Figure-Ground Assignment mechanism in some vision tasks without considering high-level supervisions, our work instead explores an end-to-end ﬁgure-ground aided approach that can be easily incorporated into existing encoder-decoder models and improve their performance.
Conclusion
In this paper, we demonstrate that the Figure-Ground cues inspired by the perceptual organization of human vision can be effectively utilized to improve the performance of a CNN model for different challenging robust perception organization tasks, e.g. , camouﬂaged object detection, polyp segmentation, and lung infection segmentation.
Speciﬁcally, we investigate the proposed FGA module, on a carefully established synthetic dataset for the Figure-Ground Segregation test, which aims to measure the capability of Figure-Ground Assignment.
The empirical study shows that the proposed new module consistently improves the performance of several representative networks in reducing visual ambiguities.
Moreover, the model implemented via our proposed FGA mechanism outperforms SOTA approaches in three challenging real-world applications.
We believe this study provides valuable insights to the community and attracts attention to exploring the Figure-Ground principles for other visual perception tasks.
Societal Impact.
Perceptual organization is one of the most challenging computer vision tasks.
Albeit being challenging, Perceptual organization is beneﬁcial to a wide range of applications [75, 83].
However, there also exists the risk that the technology is utilized in the scenario of the illegal shoot, malicious edit, and incorrect use.
Acknowledgement This work was supported by National Key R&D Program of China under Grant 2020AAA0105701, National Natural Science Foundation of China (NSFC) under Grants 62225207, 61872327 and U19B2038, and the University Synergy Innovation Program of Anhui Province under
Grants GXXT-2019-025.