Graph convolutional network‐based image matting algorithm for computer vision applications

IET Image ProcessingVolume 16, Issue 10 p. 2817-2825 ORIGINAL RESEARCHOpen Access Graph convolutional network-based image matting algorithm for computer vision applications Li Dong, Li Dong orcid.org/0000-0002-3587-0554 Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, ChinaSearch for more papers by this authorZheng Liang, Zheng Liang Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, ChinaSearch for more papers by this authorYue Wang, Corresponding Author Yue Wang wy_810818@126.com Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, China Correspondence Yue Wang, Department of Social Sciences, Shandong Medical College, Jinan 250002, Shandong Province, China. Email: wy_810818@126.comSearch for more papers by this author Li Dong, Li Dong orcid.org/0000-0002-3587-0554 Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, ChinaSearch for more papers by this authorZheng Liang, Zheng Liang Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, ChinaSearch for more papers by this authorYue Wang, Corresponding Author Yue Wang wy_810818@126.com Department of Social Sciences, Shandong Medical College, Jinan, Shandong Province, China Correspondence Yue Wang, Department of Social Sciences, Shandong Medical College, Jinan 250002, Shandong Province, China. Email: wy_810818@126.comSearch for more papers by this author First published: 13 May 2022 https://doi.org/10.1049/ipr2.12528AboutFiguresReferencesRelatedInformationPDFSectionsAbstract1 INTRODUCTION2 RELATED WORK3 METHODOLOGY4 RESULTS AND DISCUSSION5 CONCLUSION DECLARATIONS ACKNOWLEDGEMENTS CONFLICT OF INTERESTOpen ResearchREFERENCESPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessClose modalShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Image matting plays a vital role in a variety of computer vision tasks including video editing and image fusion. Previously presented image matting algorithms might fail in producing favorable results since most of them concentrate on the similarity between the neighboring pixels while neglecting the corresponding spatial relationship. To address this issue, an end-to-end image matting framework through leveraging deep learning mechanism and graph theory is proposed. The proposed pipeline is a concatenation of one deep feature extraction component and a Graph Convolutional Network (GCN). The former part takes an image and its corresponding trimap as inputs and can generate the pixel-wise features, which are then exploited as the input of the GCN locating at the latter part of the proposed framework. The GCN would refine the features for every pixel and predict the alpha matte outcome of the image. The approach outperforms a group of state-of-the-art matting techniques as shown by the theoretical analysis and experimental results in terms of both accuracy and visual effects. 1 INTRODUCTION Image matting techniques are primary procedures of image synthesis and video editing that have been widely applied in film production. Generally, image matting refers to extracting the foreground object image F and its opacity mask α (or alpha matte) from one given image I. Commonly I is founded on the linear blending of F and the background B within I with α: ◂=▸I=α⁢F+◂⋅▸(1−α)⁢B.\begin{equation} I= \alpha F+(1-\alpha )B. \end{equation}(1) Usually, there is a follow-up compositing process, in which one new image can be generated from blending a new background image as well as the extracted foreground object image F and alpha matte through leveraging (1). Therefore, image matting plays a critical role in image editing, film special-effects production and so on. Digital matting is one inherently ill-posed problem since it has more unknowns including F, B, α than the constraints (as shown in (1)). For each pixel i in the image there are 7 unknown variables (foreground RGB colors, background RGB colors of pixel i and αi$\alpha _i$) and 3 known values (RGB colors of pixel i). A large amount of algorithms have been proposed to implement image matting by adapting to the matting equation (1). In general, the image matting techniques can be divided into three categories: sampling-based [1], propagation-based [2, 3], and a hybrid of these two types [4, 5]. Among them, propagation-based algorithms [6-8], which uses the pixel-wise similarity to implement the propagation from the known alpha matte to the unknown areas, have obtained considerable achievements. However, most of the above-mentioned approaches might deteriorate in performance for the reason that (1) formulates the matting process merely as a color-related problem [9]. To be specific, their capability significantly depends on the pixel-wise intensity difference between the neighboring pixels. Thus, these algorithms including the recent deep learning-based ones [10, 11] are inevitably sensitive to the subtle overlapping region between the foreground and background components with various defect-related drawbacks. In the work of [27], Yuan, Li, and Fan proposed to compute the consistency of alpha masking to evaluate the robustness of the matting algorithms. To create an accurate trimap, Yuan and Li [29] presented an ant colony algorithm to yield the boundary information of the foreground objects. In the work of [30], a robust image matting algorithm was presented to refine the trimap generated by the image segmentation. Liu et al. [31] proposed a group optimization mechanism, whcih transform the large-scale combinatorial optimization matting into multiple small-scale issues. To address the issues caused by the intensity-based features, several deep learning approaches have been proposed. For instance, Cho et al. [10] present a deep Convolutional Neural Networks (CNN) method for natural image matting. This algorithm takes results of the closed form matting [6], the KNN matting [8] along with the inputs of normalized RGB color images, and an end-to-end mapping between the input samples and the output alpha mattes. In the work of [12] proposed by Xu et al., a two-step deep learning-based architecture is provided. In its first component, a deep convolutional encoder-decoder network is used by taking an image and the corresponding trimap as inputs and can generate the alpha matte of the image. The following part contains a convolutional network, which can refine the alpha matte predictions output from the former procedure. Recently, Wang et al. [3] propose a deep propagation-based image matting framework by introducing deep neural network into learning an alpha matte propagation mechanism. The deep matting methods can learn an alpha matte in an end-to-end fashion given an image and the trimap as input. However, almost all of them suffer from inadequate samples due to the high-dimensional parameters space of deep learning models during the training procedure. Bearing the above-mentioned analysis in mind, we propose a deep learning-based image matting pipeline with a small sample. This approach exploits a Convolutional Neural Network (CNN) along with an encoder-decoder architecture as the feature extraction instrument as well as the following Graph Convolutional Network (GCN). The semantic and low-level features can be derived from the feature extraction module by using encoder-decoder network and CNN, respectively. GCN takes the input image as a graph. Each pixel in the image along with its extracted feature is treated as a node in the graph. And the pair-wise similarity is used to form the edge between each pair of pixels. The GCN model learns hidden layer representations consisting of both globally spatial structure and features of the nodes. Therefore, it can be used for discriminating the foreground nodes from the background pixels in an input sample. To evaluate the performance of the proposed approach, we conducted comparison experiments between the state-of-the-art techniques and ours. Experimental results demonstrate that the proposed pipeline outperforms the state-of-the-arts in terms of both accuracy and the visual effects. Overall, this study offers at least the following contributions: (I) To the best of our knowledge, this work is an early work for introducing GCN into the implementation of image matting. (II) A deep learning-based feature extraction module and one GCN are integrated into an end-to-end framework that can be adapted to small sample oriented image matting scenarios. (III) The proposed approach guarantees both efficiency and effectiveness while outperforms the state-of-the-art algorithms. The remainder of this paper is organized as follows. The studies related to this study are described in Section 2. In Section 3, the details of the proposed approach are provided, and the experimental results with the discussion are reported in Section 4. Finally, we give the conclusion of this study along with the future work in Section 5. 2 RELATED WORK There are three primary types of matting methods in the literature, including sampling-based, propagation-based, and a hybrid of them. Sampling-based algorithms assume that the pixels with resembling intensities should lead to identical alpha matte estimation. Propagation-based techniques could transmit the opacity mask from user scribbles into unknown neighborhood. Both of them rely principally on intensity affinity during the generation of alpha matte values. For the sampling-based techniques [13-15], the known foreground and background areas are sampled for determining the intensity of every unknown pixel, in which a metric is employed to figure out the optimal solution. For instance, Jue Wang and Michael Cohen [13] propose a robust sampling-based matting algorithm by analyzing the confidence of the samples. Only those samples with high confidence are embraced into the matting energy function while a neighborhood regularization term is used to guarantee the smoothness of the alpha matte. In the work of [4], Gastal et al.present a real-time matting technique for both images and videos. It is built upon the observation that pixels in small neighborhood tend to share similar characteristics. By eliminating redundant foreground and background samples, the execution efficiency of this algorithm can be significantly enhanced. He et al. [5] propose a global sampling approach by using all of the available samples in an image rather than not just the neighboring pixels. On the other hand, propagation-based methods [16-19] are primarily used to avoid weak assumptions about image statistics during sampling. Their effectiveness chiefly stems from the clear mathematical functions reformulated from (1) and capable of capturing the pixel-wise similarity between the adjacent pixels. The close-form matting [6] is a well-known propagation-based algorithm, in which a cost function is derived from a prior assumption that the foreground and background intensities are locally smooth. Meanwhile, this formulation can be resolved in a close-form fashion. By treating digital matting as a semi-supervised learning task, Zheng and Kambhamettu [7] propose a local learning-based approach along with a global learning-based approach for scribble-based matting and trimap-based matting, respectively. Recently, considerable deep learning-based image matting algorithms have been presented. For instance, Shen et al. [11] propose an automated matting approach for portrait images. An end-to-end CNN is used with the portrait image as input. It does not need any user interaction, which was essential in most previous approaches. Cho et al. [10] also presents an end-to-end deep model to learn the alpha matte with the outcome of [6] and [8] as inputs. Wang et al. [3] propose a deep propagation-based image matting framework through using deep learning mechanism to learn an alpha matte propagation principle. Deep learning-based approaches have achieved superior performance over the traditional methods. However, they still suffer from the extensive amount of parameters since their training results significantly rely on the quantity of the data samples. Li and Lu [28] proposed an end-to-end approach for natural image matting with a guided contextual attention module, which can propagate high-level opacity information globally based on the learned low-level affinity. Liu et al. [32] proposed a cascaded segmented matting network, which leverages a shared encoder and two separate decoders to generate the end-to-end human image matting outcome. Ren, Li, and Song [33] presented a generative adversarial network to optimize image matting and harmonization to implement both tasks and generate more natural results. 3 METHODOLOGY Inspired by deep matting models [3, 9], we propose a DeepMatteGCNet to implement the learning of alpha matte. The motivation of this framework arises from the assumption that both the chromatic and spatial representations extracted from pixel-wise level contribute to the similarity estimation between pairwise pixels, simultaneously. To note that the similarity is determined by geometry and intensity variations together. As shown in Figure 1 DeepMatteGCNet consists of a local feature extraction module along with a global similarity optimization module. FIGURE 1Open in figure viewerPowerPoint The structure of the proposed pipeline Local feature extraction module To establish the GCN model in the proposed framework, the local feature extraction module is used to represent the pixel-wise representation. As shown in Figure 1, this feature extraction module involves two branches, including the semantic representation and the low-level feature extraction channels. The combination of the generated features from these two branches are exploited as the feature for each pixel in an input image. Note that both the input image and its corresponding manually drawn trimap are taken as the input in this stage. The architecture of the semantic-level feature extraction module follows the SegNet [25] composing of an encoder along with a decoder network, which transforms the input into feature maps by down-sampling and the features maps into output through up-sampling, respectively. To be specific, the encoder network incorporates the 10 convolutional layers and 4 max-pooling layers. Each of the convolution operator accompanied by an element-wise ReLU (rectified linear unit) generates a collection of representation maps with batch-normalization. Meanwhile, the max-pooling employs a setting with a 3 × 3 sliding window and the stride of 2. On the other hand, the decoder component then up-samples the features extracted from the encoder network through convolutional layers and up-pooling operators to produce the pixel-wise representation for each input image. Furthermore, each layer in the decoder network corresponds to its counterpart in the encoder network, for example, max-pooling and up-pooling. Similar to [25], the weighting parameters learned from the max-pooling operators in the encoder network are reused by the up-pooling layers in the decoder counterpart. Meanwhile, the low-level feature extraction channel includes 4 pairs of convolutional and ReLU layers. Proved by [3] and [9], this branch can be used to reveal more detailed information about the alpha matte. Global similarity optimization module The proposed GCN takes an adjacent matrix and one feature matrix as input. Specifically, the adjacent matrix contains the connectivity associations between each pair of pixels for an image, while feature matrix includes the pixel-wise features of the same image learned from the local feature extraction module. 3.2.1 Definition It is assumed that there are n pixels for each input image, P=◂[]▸[P1,P2,…,Pn]$P=[P_1, P_2, \ldots, P_n]$. Each image could be denoted by a matrix Ii∈◂◽˙▸Rn×mi$I_i\in \mathbb {R}^{n\times m_i}$, where m is the dimensionality for the feature vector extracted from the original image samples and ◂+▸mi∈{0,1}$m_i\in \lbrace 0,1\rbrace$. Then an image in the dataset can be represented as one weighted graph with a tuple data structure ◂=▸G=(V,E,W)$G=(V, E, W)$, where V denotes the m nodes in the graph, E represents the whole group of edges in the same graph, and W∈◂◽˙▸Rm×ni$W\in \mathbb {R}^{m\times n_i}$ is the corresponding adjacency matrix. Meanwhile, ◂◽.▸Wi,j$W_{i,j}$ is the weight assigned to the edge, which links vi∈V$v_i\in V$ to vj∈V$v_j\in V$. To note that the value of the association (the edge) denotes the similarity of the connected pixels. Therefore, it has been set as one hyper-parameter in the following experiments. The convolutional operator illustrated in Figure 2 equals to the multiplication operation in the spectral domain and relates to the common convolution operator in the time domain. The whole process could be mathematically formulated as follows: L=◂−▸Im−◂◽˙▸D−12⁢W⁢◂◽˙▸D−12,\begin{equation} L=I_{m}-D^{-\frac{1}{2}}WD^{-\frac{1}{2}}, \end{equation}(2)where L represents the Laplacian matrix, D denotes the matrix degree and Im$I_m$ is the identity matrix. FIGURE 2Open in figure viewerPowerPoint The architecture of the proposed GCN As mentioned by Defferrard et al. [20], the Laplacian matrix L then could be represented by using Chebyshev polynomials: ◂=▸Tk⁡(L)=◂⋅▸2⁢L⁢◂◽.▸Tk−1⁡(L)−◂◽.▸T(k−2),\begin{equation} T_k(L)=2LT_{k-1}(L)-T_{(k-2)}, \end{equation}(3)where ◂=▸T0⁡(L)=1$T_0(L)=1$ and ◂=▸T1⁡(L)=L$T_1(L)=L$. To note that a polynomial ordered by K can generate K filters without bias. And the filtering of a signal with K filters could be implemented with: ◂=▸o=◂⋅▸gθ⁡(L)∗c=∑k=◂⋅▸0K⁢θk⁢Tk⁡(L¯)⁢c,\begin{equation} o=g_{\theta }(L)*c=\sum _k=0_K \theta _kT_k(\bar{L})c, \end{equation}(4)where c denotes a dataset, L¯=◂−▸2◂◽.▸λm⁢a⁢x⁢L−Id$\bar{L}=\frac{2}{\lambda _{max}}L-I_d$, and ◂◽.▸λm⁢a⁢x$\lambda _{max}$ is the highest eigenvalue of the normalized L. Therefore, the output of the lth layer for each sample in a GCN can be formulated as: ocl=◂∑▸∑i=1◂◽.▸Fi⁢n◂⋅▸◂◽.▸gθil⁡(L)⁢◂◽:▸cs,il,\begin{equation} o_c^l=\sum _{i=1}^{F_{in}}g_{\theta _i^l}(L)c_{s,i}^l, \end{equation}(5)where ◂◽.▸Fi⁢n$F_{in}$ represents the input filter, ◂+▸θil∈RK$\theta _i^l\in \mathbb {R}^K$ are the Chebyshev coefficients, and ◂◽:▸cs,il$c_{s,i}^l$ is the input feature map for sample c at layer l. 3.2.2 Network architecture The structure of the proposed GCN is provided in Table 1. Totally, it is composed of 5 convolutional layers. There is no pooling operation used in the network architecture for conserving the completeness of the extracted features. The dropout rates for 2nd, 3rd, 4th, and 5th convolutional layers are 0.4. TABLE 1. Network architecture of the proposed GCN Layer Conv Conv Conv Conv Conv Conv Classifier Channels 32 32 64 64 128 128 2 K-order 9 9 9 9 9 9 N/A Stride 1 1 1 1 1 1 N/A The initial training rate of the proposed GCN is 0.001. The training is conducted with a fixed 600 steps. The learning rate would decrease by a factor of 0.5 once the validation accuracy drops in two consecutive rounds. Losses The integration of alpha prediction loss and the composition loss [9] is exploited as the loss for training DeepMatteGCNet. They are used to evaluate the Euclidean distance between the output image as well as alpha matte of DeepMatteGCNet and the corresponding ground truth, respectively. Mathematically, this process can be formulated as the following: ◂=▸L⁢o⁢s⁢s=◂+▸◂◽˙▸(α¯−α∗)2+ε+◂◽˙▸(c¯−c∗)2+ε\begin{equation} Loss=\sqrt {(\bar{\alpha }-\alpha ^*)^2+\epsilon }+\sqrt {(\bar{c}-c^*)^2+\epsilon } \end{equation}(6)where L⁢o⁢s⁢s$Loss$ represents the loss used in the training process, α¯$\bar{\alpha }$ and α∗$\alpha ^*$ denote the output alpha matte value of the proposed framework and the corresponding ground truth, respectively; c¯$\bar{c}$ and c∗$c^*$ are used to represent the composited and ground truth image colors, respectively; ε is one small number. To reduce the complexity of the computation, only the unknown pixels are taken into consideration. Implementation details The DeepMatteGCNet is implemented by leveraging Keras platform (Tensorflow-based). Both the training and testing make use of the NVIDIA Titan X graphics adapters. The training is conducted with a fixed learning rate of 0.1 and the momentum of 0.9 with the stochastic gradient descent (SGD) optimizer. And the training phase takes about 20 epochs in total. 4 RESULTS AND DISCUSSION Dataset To evaluate the performance of the proposed DeepMatteGCNet, we conducted experiments on one benchmark dataset [21] and our collected samples. Being a publicly available database, the alphamatting.com dataset [21] incorporates both the images and the corresponding alpha mattes at www.alphamatting.com. In total, it has 27 sets of images and alpha mattes along with a high-resolution and a low-resolution trimaps. Meanwhile, there are 51 images through filming 51 objects in front of a monitor that is used to display various scenes in our own database. Following the techniques presented in [3], we obtained the ground truth alpha mattes for the 51 images. To be specific, five additional backgrounds are leveraged to generate the alpha matte for each image with the solution of an over-determined linear system using singular value decomposition (SVD) [22]. For every image in our dataset, one low-resolution trimap is manually drawn. And the resolutions of all the images and trimaps are 600 × 800. Furthermore, to increase the quantity of the training samples, the original images are augmented by adding a variety of backgrounds from the indoor scene recognition database [23] as well as the outdoor dataset Places [24]. Meanwhile, 4 different rotations are applied to form the compositions of the foregrounds and alpha mattes. Totally, 257K images are obtained. In the experiments, 80% of the images in the datasets are taken as the training set and the remaining 20% are then used as the testing set. The entire training process of DeepMatteGCNet takes 4 days around. The execution time for each image is about 1.1 s in the testing process. Evaluation To evaluate the performance of the proposed approach quantitatively and visually, the comparison experiments are conducted between the state-of-the-art algorithms TransMatting, TMFNet, IamAlpha, LFPNet, TIMI-Net, and ours on both the publicly available dataset and our manually collected images. Meanwhile, both the state-of-the-arts and ours take the training set and testing set identically. As demonstrated in Figure 3 and Figure 4, the SAD (sum of absolute differences) along with MSE (mean square error) outcomes are provided over the testing set for both the public dataset and the exclusive database. According to the statistical results, the following findings can be discovered. First of all, the proposed GCN-based image matting framework outperforms the state-of-the-art techniques including the deep learning-based and non-deep learning-based ones. This could be attribute to the incorporation of both the geometric structure and chromatic information learned by the GCN, which can reveal the unique distributions for foreground and background elements in high dimensionality manifold. Moreover, the proposed approach addresses the spatial and color components in an image, separately. In contrary, the state-of-the-art algorithms resolves the intensity individually (e.g., close-form algorithm [6]) or deal with geometry and intensity features jointly (e.g., MatteCNN [10]). On the other hand, the proposed algorithm achieves close performance over the deep learning-based methods [10, 28, 33]. This might arise from the similar structures adopted by both of them and the presented approach. However, the introduction of GCN not only can guarantee the accuracy of image matting but also could reduce the requirement for deeper network architecture. FIGURE 3Open in figure viewerPowerPoint Quantitative comparison between state-of-the-arts and the proposed approach in terms of SAD and MSE on the public dataset FIGURE 4Open in figure viewerPowerPoint Quantitative comparison between state-of-the-arts and the proposed approach in terms of SAD and MSE on 8 testing images (from alphamatting.com 2022/03/14) According to the visual effects shown in Figure 5, the proposed approach also can produce overwhelmingly better outcome over the state-of-the-art algorithms, especially the overlapped areas between foreground and background with homogeneous intensity. This may benefit from processing the geometric structure and intensity information independently by introducing GCN. For instance, the "hair" area can be isolated from the contiguous "book" with similar color in Figure 5. FIGURE 5Open in figure viewerPowerPoint The visual comparison between state-of-the-arts and the the proposed GCN-based framework on three images. For each sample (top-bottom, left-right), there are the original image, trimap, ground-truth matte, mattes from Close-form [6], MatteCNN [10], DeepMatteNet [9], DeepMattePropNet [3], and DeepMatteGCNet, respectively Discussion The superior performance of the proposed framework derives from the merits as follows. On the one hand, it fully exerts the feature extraction power of CNN and GCN, simultaneously. CNN is capable of capturing the local feature for each pixel while GCN can further reveal the spatial associations between pairwise pixels accordingly. On the other hand, the features learned from the proposed CNN is suitable for the subsequent GCN module since deeper or more shallow network might provide only low-level and semantic-level representations, respectively. Furthermore, to determine the most optimal depth of the convolutional layers in the local feature extraction module, we conducted the comparison experiments with different number of layers for both the semantic-level and low-level branches (as shown in Table 2). TABLE 2. The SAD difference by using different numbers of convolutional layers Low-level Semantic-level 2 3 4 5 6 7 2 4.7 4.5 4.1 4.5 4.3 4.1 3 5.5 4.3 3.6 3.3 4.9 5.2 4 5.3 5.6 4.9 4.3 4.2 4.5 5 5.8 5.1 4.7 4.4 4.3 4.8 To note that the combination of both the encoder-decoder network and graph structure used in the proposed pipeline can improve the performance of image matting, but it also increases the computational cost of the proposed algorithm comparing with the independent encoder-decoder or graph convolutional network. 5 CONCLUSION The proposed DeepMatteGCNet considers the pixels in an image as the nodes in one graph and the similarity between pairwise pixels as the corresponding edges for each pair of nodes. Generally, it provides a novel perspective in dealing with the tasks of image matting. As elaborated by [3], the low-level and semantic-level features contribute significantly to image matting techniques. We argue that the low-level and semantic-level as well as spatial representations are vital factors to the accurate estimation of alpha matte values. This has been partially proved by the results in Table 2. Our algorithm incorporates the mechanism of the deep learning-based matting methods, which overwhelming outperforms the state-of-the-art techniques in a data-driven fashion. Different from the previously presented deep network techniques [9, 10], the presented framework in this study dose not generated the alpha matte directly. Similar to [3], our approach also need to measure the similarity between each pair of pixels before refining the connectivity relationship among the entire pixels. However, the similarity measurement component and matte propagation module in [3] have been integrated by using GCN in this study. The proposed framework achieves superior performance since it incorporates both the capability of deep image representation from CNN and the manifold learning ability from GCN. In addition, the proposed approach requires less input samples than the previous deep learning networks due to the employment of GCN [26]. Experimental results demonstrate that the superiority of the proposed framework over the state-of-the-art matting techniques. In the next phase, we would research on the extension of our framework for various image matting-related applications including image segmentation. Meanwhile, we will continue to enrich the image samples and further enhance the performance of the proposed approach with different types of pixel-wise representations and pair-wise similarity for a variety of applications. DECLARATIONS Ethical Approval and Consent to participate: Approved. Consent for publication: Approved. ACKNOWLEDGEMENTS This work was made possible through support from National Key Research and Development Program of China Sub-project (2017YFD0700405) Key R&D Project of Shandong Province (2019JZZY010734). The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions. CONFLICT OF INTEREST There is no potential competing interests in our paper. And all authors have participated in writing this manuscript and approved to submit to your journal. We confirm that the content of the manuscript has not been published or submitted for publication elsewhere. Open Research DATA AVAILABILITY STATEMENT The data that support the findings of this study are available from the corresponding author upon reasonable request. REFERENCES 1Yao, G., Zhao, Z., Liu, S., et al.: A Comprehensive Survey on Sampling–Based Image Matting. Comput. Graphics Forum 36(8), 613– 628 (2017) 2Jin, M., Kim, B., Song, W., et al.: Adaptive Propagation-Based Color-Sampling for Alpha Matting. IEEE Trans. Circuits Syst. Video Technol. 24(7), 1101– 1110 (2014) 3Wang, Y., Niu, Y., Duan, P., et al.: Deep Propagation Based Image Matting. In: International Joint Conference on Artificial Intelligence, pp. 999– 1006. Morgan Kaufmann, San Francisco (2018) 4Gastal, E.S., Oliveira, M.M.: Shared Sampling for Real-Time Alpha Matting. Comput. Graphics Forum 29(2), 575– 584 (2010) 5He, K., Rhemann, C., Rother, C., et al.: A global sampling method for alpha matting. In: Computer Vision and Pattern Recognition, pp. 2049– 2056. IEEE, Piscataway (2011) 6Levin, A., Lischinski, D., Weiss, Y., et al.: A Closed-Form Solution to Natural Image Matting. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 228– 242 (2008) 7Zheng, Y., Kambhamettu, C.: Learning based digital matting[C]. In: International Conference on Computer Vision, pp. 889– 896. IEEE, Piscataway (2009) 8Chen, Q., Li, D., Tang, C., et al.: KNN Matting. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2175– 2188 (2013) 9Xu, N., Price, B., Cohen, S., et al.: Deep Image Matting[C]. In: Computer Vision and Pattern Recognition, pp. 311–320. IEEE, Piscataway (2017) 10Cho, D., Tai, Y., Kweon, I.S., et al.: Natural image matting using deep convolutional neural networks. In: European Conference on Computer Vision, pp. 626– 643. Springer, Berlin (2016) 11Shen, X., Tao, X., Gao, H., et al.: Deep automatic portrait matting. In: European Conference on Computer Vision, pp. 92– 107. Springer, Berlin (2016) 12Xu, N., Price, B., Cohen, S., et al.: Deep Image Matting. In: Computer Vision and Pattern Recognition, pp. 311– 320. IEEE, Piscataway (2017) 13Wang, J., Cohen, M.: Optimized color sampling for robust matting. In: Computer Vision and Pattern Recognition, pp. 1– 8. IEEE, Piscataway (2007) 14Chuang, Y.-Y., Curless, B., Salesin, D.H., Szeliski, R.: A bayesian approach to digital matting. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol. 2, pp. II– 264. IEEE, Piscataway (2001). 15Shahrian, E., Rajan, D., Price, B., Cohen, S.: Improving image matting using comprehensive sampling sets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 636– 643. IEEE, Piscataway (2013) 16Chen, Q., Li, D., Tang, C.-K.: Knn matting. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2175– 2188 (2013) 17Chen, X., Zou, D., Zhiying Zhou, S., Zhao, Q., Tan, P.: Image matting with local and nonlocal smooth priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1902– 1907. IEEE, Piscataway (2013) 18Grady, L., Schiwietz, T., Aharon, S., Westermann, R.: Random walks for interactive alpha-matting. In: Proceedings of VIIP 2005, pp. 423– 429. Acta Press, Anaheim (2005) 19Lee, P., Wu, Y.: Nonlocal matting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway (2011) 20Defferrard, M., Bresson, X., Vandergheynst, P., et al.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Neural Information Processing Systems, pp. 3844– 3852. MIT Press, Cambridge (2016) 21Rhemann, C., Rother, C., Wang, J., et al.: A perceptually motivated online benchmark for image matting. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009). IEEE, Piscataway (2009) 22Chuang, Y.Y., Curless, B., Salesin, D.H., et al.: A Bayesian approach to digital matting. In: Computer Vision and Pattern Recognition, CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. IEEE, Piscataway (2001) 23Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: Computer Vision and Pattern Recognition, pp. 413– 420. IEEE, Piscataway (2009) 24Zhou, B., Lapedriza, A., Xiao, J., et al.: Learning deep features for scene recognition using places database. In: Neural Information Processing Systems, pp. 487– 495. MIT Press, Cambridge (2014) 25Badrinarayanan, V., Kendall, A., Cipolla, R., et al.: SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481– 2495 (2017) 26Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations. ICLR, Vienna (2017) 27Yuan, G., Li, J., Fan, H.: Evaluating the robustness of image matting algorithm. CAAI Trans. Intell. Technol. 5(4), 247– 259 (2020) 28Li, Y., Lu, H.: Natural image matting via guided contextual attention. Proc. AAAI Conf. Artif. Intell. 34(07), 11450– 11457 (2020) 29Yuan, G., Li, J., Hua, Z.: Image matting trimap optimization by ant colony algorithm. Mult. Tools Appl. 80(4), 6143– 6169 (2021) 30Chen, J., Li, X., Luo, L., Ma, J.: Multi-focus Image Fusion Based on Multi-scale Gradients and Image Matting. IEEE Trans. Multimedia (2021) 31Liu, S., Feng, F., Gou, H., Zhou, Z., Tan, M., Wang, L.: Grouping optimization algorithm for natural image matting. In: 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), pp. 421– 426. IEEE, Piscataway (2021) 32Liu, B., Jing, H., Qu, G., Guesgen, H.W.: Cascaded Segmented Matting Network for Human Matting. IEEE Access (2021) 33Ren, X., Li, Y., Song, C.: A Generative Adversarial Framework For Optimizing Image Matting And Harmonization Simultaneously. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1354– 1358. IEEE, Piscataway (2021) Volume16, Issue1021 August 2022Pages 2817-2825 FiguresReferencesRelatedInformation RecommendedFully automatic figure‐ground segmentation algorithm based on deep convolutional neural network and GrabCutRuigang Fu, Biao Li, Yinghui Gao, Ping Wang, IET Image ProcessingNovel superpixel‐based algorithm for segmenting lung images via convolutional neural network and random forestCaixia Liu, Mingyong Pang, Ruibin Zhao, IET Image ProcessingImage segmentation using multilevel graph cuts and graph development using fuzzy rule‐based systemMuhammad Rizwan Khokher, Abdul Ghafoor, Adil Masood Siddiqui, IET Image ProcessingForeground–background decoupling mattingJiawei Wu, Guolin Zheng, Kun Zeng, Haoyi Fan, Zuoyong Li, International Journal of Intelligent SystemsImage segmentation algorithm based on superpixel clusteringLin Cong, Shifei Ding, Lijuan Wang, Aijuan Zhang, Weikuan Jia, IET Image Processing