CAPNet: Cartoon Animal Parsing with Spatial Learning and Structural Modeling

Authors: jian-jun  qiao
yu  duan
xiao  wu
wei  li
wei li concat
matmul  softmax
conv conv conv
d c conv
conv  ca
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-27 03:06:52

Abstract: Cartoon animal parsing aims to segment the body parts such as heads, arms, legs and tails from cartoon animals. Different from previous parsing tasks, cartoon animal parsing faces new challenges, including irregular body structures, abstract drawing styles and diverse animal categories. Existing methods have difficulties when addressing these challenges caused by the spatial and structural properties of cartoon animals. To address these challenges, a novel spatial learning and structural modeling network, named CAPNet, is proposed for cartoon animal parsing. It aims to address the critical problems of spatial perception, structure modeling and spatial-structural consistency learning. A spatial-aware learning module integrates deformable convolutions to learn spatial features of diverse cartoon animals. The multi-task edge and center point prediction mechanism is incorporated to capture the intricate spatial patterns. A structural modeling method is proposed to model the complex structural representations of cartoon animals, which integrates a graph neural network with a shape-aware relation learning module. To mitigate the significant differences among animals, a spatial and structural consistency learning strategy is proposed to capture and learn feature correlations across different animal species. Extensive experiments conducted on benchmark datasets demonstrate the effectiveness of the proposed approach, which outperforms state-of-the-art methods.

Main paper: 
 related work 2.1 human parsing: 
 The human parsing methods [4, 18, 31] have made significant progress in recent years, with numerous approaches proposed to parse body parts of humans. Spatial Learning. Spatial information plays a crucial role in parsing tasks by providing crucial information about the relative positions. It facilitates the recognition of complex and ambiguous regions by considering the spatial distribution of the human body, achieving more accurate parsing results. Previous methods [22, 42] enhance the learning ability of spatial patterns by embedding a pose estimation branch into the human parsing framework. MMAN [21] improves the capability of capturing consistent spatial information through local and global learning. CDGNet [20] learns the spatial characteristics of the human body in horizontal and vertical directions to improve the performance of human parsing. Overall, the integration of spatial information enhances the performance of human parsing algorithms.Structural Modeling. In addition to spatial features, structural information is also important for human parsing. To transcend the limitations of conventional CNN-based approaches, previous works have proposed structural modeling modules. These existing approaches have integrated the prowess of convolutional neural networks (CNNs) [12, 30] and graphical models [13, 19, 23, 28] to extract visual features and model structural information. Previous methods [35, 36] introduce graph neural networks (GNNs) to model the regular structure of human body, leveraging the powerful structural reasoning ability of GNNs to achieve promising results in human parsing. Some methods [17, 44] decompose and model the human body with attention mechanisms [5, 15, 16, 37, 38, 40] to learn and capture important structural information, improving the performance of human parsing. However, these techniques are designed for human parsing and they have limited performance when facing the cartoon animal images with irregular body structures, complex visual appearances, abstract drawing styles, and so on.In recent years, there has been a growing interest in the field of cartoon parsing. The traditional cartoon parsing approach [39] segments cartoon images into different regions using conventional linear iterative clustering superpixels and adaptive region propagation merging techniques. However, it lacks semantic discrimination for each segmented region. Recently, pioneering works [26, 33] have adapted deep learning-based human parsing techniques to the realm of anthropomorphic cartoon characters and cartoon dogs, achieving commendable results. DFPNet [33] designs a dense multiscale pyramid network to capture multi-scale information of cartoon dogs, addressing the scale issue in cartoon dog parsing. CPNet [26] proposes a pixel and part correlation method to address the irregularities of cartoon characters. By learning pixel-level and part-level correlations, it identifies complex cartoon structures and improve the performance of cartoon parsing. However, these methods have limitations since they primarily focus on human-like forms or single-category cartoon characters such as dogs, ill-equipped to appearance diversity and structural complexity in different cartoon animals like alligators and butterflies. To alleviate the challenges of cartoon animal parsing, a novel spatial learning and structural modeling network, named CAPNet, is proposed to integrate spatial perception, structure modeling, and spatial-structural consistency learning. The framework is illustrated in Fig. 2 . ConvNet [12] and Pyramid Pooling Module (PPM) [46] are used to extract convolutional features. The spatial learning branch introduces deformable convolutions to learn spatial information related to the irregular body parts of cartoon images. It is augmented by multi-task edge and center point prediction strategy, which captures the edge-aware and center-aware spatial features. The structural modeling branch employs a graph neural network to model the complex cartoon animal structures. It designs a body part relation learning network based on shape-aware convolution and self-attention mechanism, which captures structural shape information and associates the node features of the body parts. The captured spatial features and structural representations are seamlessly integrated with cross-attention mechanism, which achieves consistency among the complex and diverse cartoon animals. 
spatial learning: 
 The spatial information is of great importance for cartoon animal parsing. The distributions of the center points and edges of the body parts are critical to delineate the spatial patterns of cartoon animal images, which are illustrated in Fig. 3 . To learn the intricate spatial patterns, CAPNet integrates deformable convolutions to transcend the rigid receptor fields of conventional convolution, capturing spatial features of abstract cartoon images. It incorporates multi-task edge and center point prediction strategy to capture edge-aware and center-aware spatial features, which makes the network adapt to the distortions and abstraction in spatial dimension. The framework of the proposed spatial learning is illustrated in Fig. 4 , in which the middle branch with deformable convolutions is adopted to extract spatial features of irregular body parts. The edge prediction module aims to capture discriminative representations of contours, while the center point estimation part provides spatial cues that facilitate the localization and delineation of body parts.Formally, given features 𝑋 ∈ R 𝐶 ×𝐻 ×𝑊 generated by PPM module [46] , 𝑋 𝑑 is the output of the deformable convolutions with 𝑋 as input. 𝐶 refers to the number of channels, 𝐻 and 𝑊 denote height and width of the feature maps. The edge features 𝑋 𝑒 are generated by an edge predictor consisting of 5 convolutional layers, with shallow and deep features of the backbone used as inputs [18, 27] . To generate center-aware spatial features 𝑋 ℎ , 𝑋 is fed into a heat map decoder consisting of 2 deconvolution layers. 𝑋 𝑑 is then fed into a channel attention module [15] , aiming to build the connections of feature maps in 𝑋 𝑑 . The channel attention-based refinement bridges the gap among the edge prediction task, the center point prediction task, and the parsing task. It is formulated as,EQUATION𝑋 𝑐 𝑑 = 𝑋 𝑑 ⊕ 𝑊 𝑐 ⊙ 𝑋 𝑑 , 𝑋 𝑐 𝑒 = 𝑋 𝑒 ⊕ 𝑊 𝑐 ⊙ 𝑋 𝑒 , 𝑋 𝑐 ℎ = 𝑋 ℎ ⊕ 𝑊 𝑐 ⊙ 𝑋 ℎ (2)where 𝑊 𝑐 denotes channel attention weights. 𝑆𝑖𝑔(•) denotes sigmoid operation. 𝐶𝑜𝑛𝑣 (•) means convolution operation. 𝑀𝑝 (•) refers to global max pooling. ⊕ and ⊙ denote element-wise addition and multiplication, respectively.The proposed approach leverages a spatial attention mechanism [38] to effectively integrate the complementary spatial features from the edge prediction and center point prediction branches. The enriched spatial features are transferred to the parsing branch. This enables the parsing branch to obtain discriminative spatial cues captured by the auxiliary tasks. The synergistic integration of multi-task spatial features, mediated by attention-based fusion, helps the network learn important spatial cues in cartoon images. It is formulated as follows:EQUATIONEQUATIONEQUATIONwhere 𝑊 𝑠𝑒 and 𝑊 𝑠ℎ indicate the spatial attention weights from edge-aware features and center-aware features, respectively. 𝐶𝑎𝑡 (•) denotes concatenation operation. 𝑀𝑎𝑥 (•) and 𝐴𝑣𝑔(•) are the functions aggregating salient spatial features by computing max and average values alongside the channel dimension, respectively. Finally, the edge features 𝑋 𝑐 𝑒 , the center point features 𝑋 𝑐 ℎ , the edge-aware parsing features 𝑋 𝑠𝑝 𝑒 and the canter-aware parsing features 𝑋 𝑠𝑝 ℎ are integrated to obtain the spatial-aware parsing features. It is formulated as,EQUATIONThrough the integration of deformable convolutions and multitask predictions, the network learns spatial awareness. This is indispensable to address the parsing problem caused by the complex spatial distributions of cartoon animals.The structural information, including the structural shapes and relations of the body parts, is also important for cartoon animal parsing. To model the complex structural features that characterize cartoon animals, the structural modeling module is designed. CAPNet designs a shape-aware graph neural network to model graphical representations of cartoon animals. The graph neural network is adopted to construct the structures of cartoon animals and capture the relations among the body parts. The shape convolutions are utilized to capture shape information for the graph nodes. Moreover, a self-attention mechanism is adopted to associate the graph nodes with shape information.The framework of the structure modeling is illustrated in Fig. 5 . CAPNet leverages the shape-aware GNN to construct and refine a graphical representation G = (V, E), where V denotes the set of nodes corresponding to body parts, and E encodes the structural relationships of these nodes. The shape-aware GNN operates by updating the node representations of the body parts through a message passing scheme.In the structure modeling, 𝑋 is processed to node features with a convolutional layer to adjust channel dimension and a split operation to split the features into node features𝑋 𝑛𝑜𝑑𝑒 ∈ R | V | ×𝑐 ×𝐻 ×𝑊 .|V | denotes the number of nodes and |V | × 𝑐 = 𝐶. The adjacency matrix is calculated to establish connections between neighboring semantic parts. CAPNet leverages Message Passing Neural Network (MPNN) [9] to model the correlations among nodes. MPNN is structured into two stages: message passing and readout. CAP-Net designs a shape-ware relation network as the message-passing function for cartoon animal features, aiming to connect different nodes with the guidance of shape information. For every node 𝑣, the message passing gathers messages 𝑚 𝑣 from its neighbors N 𝑣 , which is formulated as,EQUATIONwhere 𝑋 𝑣 𝑛𝑜𝑑𝑒 and 𝑋 𝑢 𝑛𝑜𝑑𝑒 are elements of 𝑋 𝑛𝑜𝑑𝑒 . 𝑀 (•) is the message function used to collect part relations. 𝑀 (•) is formulated as follows:EQUATIONwhere 𝜙 denotes the shape-ware relation learning network. The proposed shape-aware relation learning network leverages the information association capability of self-attention, which captures the long-range dependencies of the body parts. It distinguishes itself by incorporating shape-aware convolutions to perceive geometric shapes and utilizing prior geometric information. By combining self-attention with shape-aware convolution and geometric priors, the method aims to capture both long-range dependencies and salient structural representations effectively.The framework of the shape-aware relation network is illustrated in Fig. 5 , which takes the graph nodes as inputs. The graph nodes are fed into three shape-aware convolutions with kernel size of 3, which aims to capture shape information. The vanilla convolution layer is defined as,EQUATIONwhere 𝑋 𝑢,𝑣 𝑐𝑎𝑡 = 𝐶𝑎𝑡 [𝑋 𝑢 𝑛𝑜𝑑𝑒 , 𝑋 𝑣 𝑛𝑜𝑑𝑒 ].𝑊 𝐾 denotes the learnable weights of kernels in a convolution layer.CAPNet employs the shape-aware convolution [1] to replace the vanilla convolution in the shape-aware relation network, which aims to capture shape information. It is formulated as,EQUATIONwhere 𝑆𝐶 (•) denotes shape convolution. and * are base-product and shape-product operator [ The shape-aware structure learning process is defined as,EQUATION)where 𝐺 is initialized and learned as [14] . The difference is that the geometric priors of CAPNet are applied for the global feature maps.𝐴 is the attention maps calculated from 𝑋 𝑄 𝑠𝑐 and 𝑋 𝐾 𝑠𝑐 [8, 32] . With the gathered message 𝑚 𝑣 , the node feature 𝑋 𝑣 𝑛𝑜𝑑𝑒 is updated as,EQUATIONwhere 𝐶𝑜𝑛𝑣𝐺𝑅𝑈 [29] is adopted as update function. In the readout process, the node features are combined and projected into segmentation features as,EQUATIONBy integrating the GNN-based structure modeling and shapeaware relation learning, the network captures the intricate structural information of diverse cartoon animals. This is of great importance to delineate the complex and diverse body parts.The body parts of cartoon animals exhibit substantial variations across different animal categories, which are usually caused by the complex and inconsistent spatial and structural characteristics. Therefore, a novel spatial-structural consistency learning (SSCL) mechanism is proposed to seamlessly integrate the spatial and structural features, achieving consistency among the spatial and structural dimensions. SSCL employs the cross-attention mechanism to correlate the spatial and structural features. It learns a consistent feature representation by fully utilizing the complementary spatial and structural information. Specifically, the spatial information is employed as the queries, while the structural information serves as the keys. The rationale behind this design is to exploit the querykey matching mechanism, which aims to learn the correspondences between spatial and structural features. For diverse cartoon animals, their spatial and structural characteristics may vary significantly. The cross-attention mechanism can adaptively adjust the fusion weights between these two modalities, effectively capturing the correlations between spatial and structural information. This results in a more consistent feature representation that better encapsulates the holistic characteristics of the cartoon animals.In the consistency learning, the spatial features 𝑋 𝑠𝑝 are treated as the queries, the structural features 𝑋 𝑠𝑡 are used as the keys, and the concatenated features 𝑋 𝑐𝑎𝑡 = 𝑐𝑎𝑡 [𝑋 𝑠𝑝 , 𝑋 𝑠𝑡 ] as the values. The consistency learning is formulated as follows:EQUATION𝑋 𝑓 𝑢𝑠𝑒𝑑 = 𝐶𝑟𝑜𝑠𝑠𝐴𝑡𝑡𝑒𝑛(𝑄, 𝐾, 𝑉 )where 𝑊 𝑞 ,𝑊 𝑘 ,𝑊 𝑣 are learnable weights from convolution layers. Through this aggregation mechanism, a fused feature representation 𝑋 𝑓 𝑢𝑠𝑒𝑑 that incorporates consistent spatial and structural information is obtained.The objective function of this paper consists of parsing loss, center point loss and edge loss. The parsing loss is set to predict the segmentation results. The center point loss is used to predict the center points of the body parts. The edge loss is utilized to predict the edges of the body parts. The objective function is defined as,EQUATIONwhere L 𝑒𝑑𝑔𝑒 [20, 27] denotes the weighted cross-entropy loss between the detected edge map and the binary edge label map. The parsing loss is defined as,EQUATIONwhere 𝑔 𝑖𝑛 denotes the ground truth label of the 𝑛-th body part on the 𝑖-th pixel and 𝑝 𝑖𝑛 is the related prediction result. 𝑀 is the number of pixels in the image. 𝑁 is the number of classes. The center point loss [10, 26] is defined as,EQUATIONwhere 𝐻 𝑛 (𝑥, 𝑦) represents the predicted result for the 𝑛-th center point of the 𝑛-th body part at the pixel location (𝑥, 𝑦). 𝐺 𝑛 (𝑥, 𝑦) is the ground truth. The proposed method alleviates the problems caused by the complexity and diversity in the cartoon animal domain with the above objective function. 
experiments 4.1 datasets and evaluation metrics: 
 CASet. To evaluate the proposed methods, a cartoon animal parsing dataset called CASet is collected from the Internet and annotated. It contains 2,643 images of cartoon animals. The dataset is divided into 1,718 images for training, 262 images for validation and 663 images for testing. The images are annotated according to different body parts, which include 9 categories: head, body, wing, tail, left arm, right arm, left leg, right leg, and a background class. The dataset comprises a wide range of cartoon animal categories, including 52 classes such as ant, horse, monkey and penguin. Each category contains dozens of cartoon images. The 52 animal categories are listed as follows: alligator, antelope, ant, bee, butterfly, Following previous studies [20, 25, 26, 33] , Mean Intersection over Union (Mean IoU) is adopted as the evaluation metric. It computes the average intersection-over-union ratio between predicted body parts and ground truth. Moreover, Mean Accuracy (Mean Acc.) is utilized for per-class accuracy calculation, and Pixel Accuracy (Pixel Acc.) measures the accuracy of correctly predicted pixels.To implement the proposed CAPNet, a ResNet-101 backbone [12] with PPM [46] and DeepLabv3+ decoder [3] is used as the baseline model. Stochastic Gradient Descent (SGD) is employed as the optimizer with a momentum of 0.9 and a weight decay of 5e-4. Following previous works [2, 8, 24, 43] , CAPNet employs the "poly" learning rate strategy, defined as 𝑙𝑟 = 𝑙𝑟 𝑖𝑛𝑖𝑡 × (1 -𝐶 𝑛 𝑇 𝑛 ) 𝑝𝑜𝑤𝑒𝑟 , where 𝑙𝑟 and 𝑙𝑟 𝑖𝑛𝑖𝑡 are the current learning rate and the base learning rate, respectively. 𝑝𝑜𝑤𝑒𝑟 = 0.9. 𝐶 𝑛 and 𝑇 𝑛 represent the current iteration number and the total iteration number, respectively. For data augmentation, CAPNet applies random left-right flipping with a 0.5 probability and random scaling strategy.For a fair comparison on CASet, all methods are trained with a batch size of 8 and image size of 384×384. The number of training epochs is 150. For the compared methods, other train settings are kept the same as their original papers. For CartoonSet, CAPNet is trained with 150 epochs. The learning rate is 7e-3 and the training size of images is 384×384. For Cartoon dog dataset, CAPNet uses a [18, 20, 36] for the LIP dataset.To evaluate the performance of the proposed method, it is compared with the state-of-the-art cartoon parsing and human parsing methods. The comparison results are listed in Table 1 . The general semantic segmentation method DeepLabV3+ [3] lacks dedicated modules for cartoon animal parsing. Accordingly, its performance is limited. Human parsing methods such as CDGNet [20] , CE2P [27] , and SCHP [18] focus on spatial information extraction and label noise learning, which improves the accuracy of cartoon animal parsing. But they ignore the modeling and learning of structural information, which limits the performance. HHP considers structural information modeling but overlooks spatial information learning. Moreover, when capturing spatial and structural information, they mainly consider the simple spatial patterns and regular structures of the human body, without taking into account the abstract and irregular properties of cartoon animals. DFPNet [33] is used for cartoon dog parsing with a dense pyramid learning structure to address the multi-scale problem in cartoon parsing. But its performance is limited when facing the challenges posed by cartoon animal parsing. CPNet [26] improves the segmentation of the irregular structures of cartoon characters using pixel and part correlation learning strategy. However, it focuses on independent cartoon characters and has limited generalization ability when segmenting diverse cartoon animals. Overall, these existing methods ignore the intrinsic differences between cartoon animals and real-world humans.The distinctions between cartoon animals and common cartoon characters are also not noticed. As a result, they have difficulties in cartoon animal parsing. To address the challenges in cartoon animal parsing, CAPNet captures the important spatial information and complex structural context by introducing the spatial learning method and structural modeling approach. It deeply explores the essential characteristics of cartoon characters by associating the spatial and structural features, achieving consistency among diverse and complex cartoon animals. The proposed method achieves the highest results on CASet, outperforming the compared methods. Table 2 gives the comparison results on the CartoonSet dataset. CartoonSet is a cartoon parsing dataset containing diverse and multi-style cartoon characters. Previous methods have a similar trend as in CASet. Overall, approaches using conventional deep learning methods have relatively worse performance, for example, DeepLabV3+ [3] . When structural representations are captured with GNN, the performance is boosted, for example, HHP [36] , CNIF [35] and CPNet [26] . Methods like CE2P [27] and CDGNet [20] incorporate the spatial information, including edges and class distributions, which further improves the performance of cartoon parsing. But when facing the irregularity and complexity of cartoon characters, the performance of previous methods is limited. CAPNet outperforms the state-of-the-art methods by learning the important spatial cues like center points and contours, and crucial structural shapes and relations of the body parts.The results of different methods on the Cartoon dog dataset are listed in Table 3 . The Cartoon dog dataset contains cartoon dogs of diverse styles and appearances, for which traditional segmentation methods like Mask R-CNN [11] and human parsing approaches like CE2P [27] are not well-suited. DFPNet [33] is proposed specifically for cartoon dog parsing and improves the results, but it primarily focuses on multi-scale feature learning. The performance is boosted by CDGNet [20] with its class distribution learning module. SCHP [18] further improves the results by proposing a self-correction strategy to correct misclassified regions. CPNet [26] learns the irregularity of cartoon characters, achieving better results. However, these methods still overlook the unique spatial and structural characteristics of diverse cartoon dogs, resulting in limited performance. Compared to these methods, the proposed CAPNet leverages the essential characteristics of cartoon dogs by focusing on the diversity and complexity of cartoon images, which achieves the state-of-theart parsing results on the Cartoon dog dataset.To evaluate the generalization ability of the proposed method for human parsing, the comparison of different methods on LIP dataset is listed in Table 4 . LIP is a large-scale human parsing dataset that is widely used [18, 20] . DeepLabV3+ [3] , OCR (ResNet101) [41] , HRNetV2 [34] and OCR (HRNetV2-W48) [41] are common semantic segmentation methods. They have limited performance on LIP dataset due to the lack of specific components for human parsing. HHP [36] , SCHP [18] , CDGNet [20] and similar approaches DeepLabV3+ [3] n/a n/a 52.09 CE2P [27] 87.37 63.20 53.10 CorrPM [45] 87.68 67.21 55.33 OCR (ResNet101) [41] n/a n/a 55.60 HRNetV2 [34] n/a n/a 55.90 OCR (HRNetV2-W48) [41] n/a n/a 56.65 CPNet [26] 88.29 68.41 57.21 CNIF [35] 88.03 68.80 57.74 HHP [36] 89.05 70.58 59.25 SCHP [18] n/a n/a 59.36 CDGNet [20] 88 are designed with specific modules tailored for human parsing, achieving good performance in human parsing. Although CAPNet is proposed for cartoon animal parsing, it achieves competitive results on LIP dataset, outperforming the cartoon parsing method CPNet [26] and most of the human parsing methods. The general semantic segmentation methods such as HRNetV2 [34] and OCR (HRNetV2-W48) [41] , although have good performance on generic datasets like Cityscapes [6] , exhibiting limited results on the human parsing task. Compared to these common segmentation methods, CAPNet, which is designed for cartoon animal parsing, achieves better results on LIP dataset. The competitive performance of CAPNet on LIP dataset suggests that the proposed method is effective for human parsing as well and demonstrates its generalization ability from cartoon animal parsing to human parsing.To validate the effectiveness of each module in CAPNet, ablation studies are conducted. Table 5 gives the experimental results of the proposed three modules: SLB (Spatial Learning Branch), SMB (Structural Modeling Branch), and SSCL (Spatial-Structural Consistency Learning). The baseline model, which lacks specialized designs for cartoon animal parsing, has limited performance on CASet dataset. When SLB module is added to learn spatial patterns, a significant performance boost is obtained, which proves the importance of spatial learning. With the SMB adopted, the performance is substantially improved, which demonstrates the crucial role of structural shape and relation modeling. The performance is further enhanced by integrating the spatial features and structural contexts, which indicates the complementary nature of these two types of information to collectively improve cartoon animal parsing. SSCL is proposed to learn the consistency between spatial cues and structural representations, which results in an additional performance gain. This suggests that the integration and consistency learning of the spatial and structural features is essential to address the challenges in cartoon animal parsing. To effectively capture the spatial patterns and structural representations of cartoon animals, CAPNet employs deformable convolution [7] , shape-aware convolution [1] and geometry prior [14] . Ablation studies are further conducted to demonstrate the effectiveness of these components. The results of the ablation experiments are listed in Table 6 . As can be seen from the table, CAPNet without the deformable convolution, shape-aware convolution or geometry prior leads to a decline in performance, which indicates that these components are effective for the network to capture and learn spatial distributions and structural contexts. Deformable convolution, with its unique convolution design, enables the network to learn irregular shapes and structures. It enhances the ability of CAPNet to capture spatial patterns of cartoon images. Shape-aware convolution helps the network capture the shape information of the body parts, which boosts the structural representation learning of CAP-Net. The geometry prior strengthens the perception and learning of the geometric structure of cartoon animals. The integration of these components contributes to the state-of-the-art performance of CAPNet in cartoon animal parsing by strengthening the learning and modeling of the important spatial cues and structural contexts.Fig. 7 showcases the parsing results of different methods on CASet dataset. Compared to other methods, the proposed method has better performance in segmenting various and intricate body parts of cartoon animals such as crab and deer. The results demonstrate that the proposed method has good generalization ability across different styles and species of cartoon animals. The performance The visualization results of the proposed modules are illustrated in Fig. 8 . Due to the significant differences and complex structures of the cartoon animals, the baseline model struggles to segment the body parts. By incorporating the spatial learning and structural modeling method, the parsing results of cartoon animals are noticeably improved. However, simply combining the spatial learning and structural modeling still leads to limited performance. By leveraging the spatial-structural consistency learning to learn the spatial and structural consistency, the performance of the network is further improved, leading to more accurate parsing results for cartoon animals.Although CAPNet achieves promising performance for cartoon animal parsing, there is still some space to improve. Currently, it focuses on spatial learning and structural modeling of cartoon animals. But it ignores the interference and influence of complex backgrounds. When the backgrounds have similar colors, visual appearances or textures to cartoon animals, CAPNet has difficulties in capturing and learning the relationship between cartoon animals and the complex backgrounds. As a result, it cannot distinguish cartoon animals from complex backgrounds. Some failure examples are illustrated in Fig. 9 , which illustrate the limitations of CAPNet. 
conclusion: 
 In this paper, a spatial learning and structural modeling network, named CAPNet, tailored for cartoon animal parsing is introduced. It addresses the challenges in spatial information learning, structural representation modeling, as well as spatial-structural consistency learning in cartoon animal parsing. The spatial-ware learning structure utilizes deformable convolutions to capture spatial features of irregular body parts. In addition, it predicts edges and center points of semantic parts to learn edge-aware and center-aware spatial patterns. The structural modeling approach incorporates GNN and shape-aware relation learning network to model intricate structures. The spatial and structural consistency learning module enhances feature representation consistency among dissimilar cartoon animals, which demonstrates improved parsing accuracy. In the future, the proposed method will be improved to address the problems caused by complex backgrounds. The dataset will be expanded to include more animal categories with more diverse and intricate backgrounds. 
ant: 
 Butterfly Vulture Scorpion Sheep Giraffe Fish Cartoon characters are important components of various multimedia applications with their vivid, attractive and imaginative expressive capabilities. The cartoon-centric applications include the metaverse, animated films, virtual reality and artistic creations. A major category of cartoons is animals, which consist of a wide range of diverse species such as reptiles, birds, mammals and fish. They typically exhibit diverse, rich and complex body parts, such as ears, limbs, wings and claws. Cartoon animal parsing, an emerging frontier of cartoon animals, seeks to semantically delineate various body parts. It enhances the comprehension of cartoon characters and makes progress in cartoon-centric applications. Unfortunately, previous cartoon parsing methods are proposed to parse anthropomorphic cartoon characters [26] or single animal category like cartoon dog [33] , which exhibit more visual and structural consistency. On the contrary, cartoon animals have significant diversity in visual appearances and structures. As a result, techniques of cartoon parsing have unsatisfactory performance in the domain of cartoon animal parsing, which motivates this work.Cartoon animal parsing is a challenging task due to the irregular structures, complex appearances, abstract styles and various animal categories, which is illustrated in Fig. 1 . Compared to real-world humans or animals, the body parts in cartoon animals usually demonstrate inconsistent spatial and structural attributes. This includes complex and uncertain spatial distributions of the center points and edges of the body parts, and irregular structural shapes and sizes of the semantic parts. The spatial and structural differences among diverse cartoon animals are conspicuous even within the animals of the same category. More specifically, the body parts of real-world humans or animals possess consistent spatial and structural attributes. The spatial patterns such as center point positions and edge contexts of their body parts, or the structure properties like the shapes and sizes of their semantic parts, are usually consistent. However, when facing cartoon animals, the spatial distributions of the center points and edges of the body parts, or the structural shapes and sizes of the semantic parts, are not consistent due to the abstract properties of cartoon images. Therefore, cartoon animal parsing remains a challenging task.Conventional human parsing methods have good performance in extracting visual features and predicting pixel-wise labels for human bodies. Although some approaches have incorporated attention mechanisms and graphical models to capture important spatial cues and structural representations, they are designed for the realworld human bodies, which have limited performance when facing the abstract cartoon images. Cartoon parsing methods are proposed to address the problems caused by the scale variations [33] and irregular body structures [26] of cartoon images. Unfortunately, the spatial learning and structural modeling of cartoon animal parsing have not been fully explored.In this paper, a novel spatial learning and structural modeling network, named CAPNet, is proposed to alleviate the challenging problems of cartoon animal parsing. The framework is illustrated in Fig. 2 . The cartoon animal image is fed into ConvNet [12] followed by Pyramid Pooling Module (PPM) [46] to extract contextual features. The spatial learning structure is designed to capture the spatial patterns of the body parts, with deformable convolutions to learn spatial features of irregular body parts. In addition, it predicts the edges and center points of the body parts to obtain edge-aware and center-aware spatial features. The structural modeling module is designed to capture the intricate structures of cartoon animals. A graph neural network (GNN) is employed to model the complex structures of cartoon animals. A shape-aware relation learning network is designed to learn the relations of the node features of GNN, which considers the shape information during the structural modeling. To integrate the spatial and structural features and achieve consistency among diverse cartoon animals, a spatial-structural consistency learning strategy based on cross-attention mechanism is proposed. With the novel structures, CAPNet alleviates the challenges of spatial learning, structure modeling, and spatial-structural consistency learning in cartoon animal parsing. Experiments conducted on the cartoon animal parsing and cartoon parsing datasets demonstrate the effectiveness of CAPNet, which outperforms the state-of-the-art approaches. The contributions are listed as follows:• A novel spatial learning and structural modeling network named CAPNet is proposed for cartoon animal parsing, which addresses the challenges of spatial learning, structure modeling, and spatialstructural consistency learning. • To capture and learn the complex and inconsistent spatial patterns in diverse cartoon animals, a spatial learning branch with deformable convolutions and a multi-task prediction strategy is designed, which enhances the spatial awareness of the network. • To model the structural information and capture the intricate structural relationships within cartoon animals, a structural modeling branch is proposed with a graph neural network and a shape-aware relation network. • A spatial-structural consistency learning strategy is proposed with a cross-attention mechanism, which aims to achieve consistent learning among spatial and structural features and mitigates the problems caused by the complexity and diversity of cartoon animals. • The proposed method achieves state-of-the-art performance on the cartoon parsing datasets, which demonstrates the effectiveness of the proposed spatial learning and structural modeling network. 
 
 Appendix:  


Decision: This paper focuses on the cartoon animal parsing task, which is challenging due to the irregular body structures and diverse categories. The topic is very interesting, and the paper provides a thorough evaluation of the network. The qualitative results look promising, and the designed two-branch architecture is reasonable. Although there are some typos, they have been well addressed and do not affect the overall readability. The paper ultimately received scores of Borderline Accept, Borderline Accept, and Weak Accept. Therefore, I recommend this paper to be accepted as a poster for the MM conference.

Reviews: review: paper_summary: paper_summary: The paper introduces CAPNet, a network designed for the task of cartoon animal parsing, which involves segmenting cartoon animal images into distinct body parts like heads, legs, and tails. Unlike existing methods, CAPNet tackles unique challenges posed by cartoon imagery, such as irregular body structures and abstract styles. The network combines spatial learning with structural modeling through a multi-pronged approach. It incorporates deformable convolutions to adapt to the varied shapes of cartoon characters and utilizes a graph neural network alongside a shape-aware relation learning module for understanding complex structural relationships. Extensive experiments demonstrate that CAPNet outperforms existing methods on benchmark datasets.
strength_weakness: strength_weakness: - The topic of cartoon segmentation is intriguing.
- The paper provides a thorough evaluation of the network, including comparisons with state-of-the-art methods across various datasets.
- The paper is generally easy to follow.
limitations: limitations: - In Eq. 8, are there any balancing factors in the loss function? If so, how to determine such hyperparameters?
- It seems to me that the training dataset is quite small-scale. Is there any over-fitting issue occurred during training?
- The improvement over other approaches seems not significant, which could be understandable given the nature of cartoon images. 
- It would be great if the authors could include some failure cases and more limitation discussions.
paper_summary: paper_summary: This paper focuses on the task of cartoon animal parsing.
strength_weakness: strength_weakness: 1. The paper is well-written, with the equations, tables, and figures appropriately arranged. 
2. The qualitative results look promising; for instance, the segmentation of the wolf head in Figure 7 is very precise and error-free.
limitations: limitations: Cartoon animal parsing is a valuable research direction with significant application potential. I noticed that the experiments in the paper (including the supplementary material) are based on a specific dataset. I wonder if the authors could provide segmentation results on well-known cartoon characters, applying their method to more general and challenging environments such as Peppa Pig, Pokemon, or even anime human characters. Demonstrating the method's effectiveness in these broader contexts could showcase its wider applicability and value.
paper_summary: paper_summary: This paper focuses on the cartoon animal parsing task, which is difficult since body structures can be irregular and the category can be diverse. To tackle these issues, this paper proposes a new approach with spatial learning and structural modeling. Moreover, the multi-task paradigm is deployed to explore the power of edge and center point detection. Experimental results confirm the effectiveness of the proposed approach.
strength_weakness: strength_weakness: 1.	The analysis of the cartoon animal parsing task, including its difficulty and key problems, is well explained in detail, which can provide a good reference for the community.
2.	The designed two branches of architecture, including the spatial learning branch and the structural modeling branch, can serve different roles and provide mutual help for each other, which is also validated by the ablation study.
3.	Experimental results show that the proposed method can achieve the best performance among four challenging datasets.
limitations: limitations: 1.	Minor: the template used by this submission is not the latest one. The MM official updated it before the final paper submission deadline.
2.	The limitations of this newly proposed method and future work are not presented.
3.	It is recommended to provide experimental results of different methods’ generalization ability evaluation. Like training on dataset 1 and evaluating on dataset 2 without finetuning.
4.	It is recommended to keep the abbreviation of Table and Figure the same. 
5.	Please consider releasing the code for the community.