Uncertainty-Aware Pseudo-Labeling and Dual Graph Driven Network for Incomplete Multi-View Multi-Label Classification

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-25 09:20:07

Abstract: Multi-view multi-label classification has recently received extensive attention due to its wide-ranging applications across various fields, such as medical imaging and bioinformatics. However, views and labels are usually incomplete in practical scenarios, attributed to the uncertainties in data collection and manual labeling. To cope with this issue, we propose an uncertainty-aware pseudo-labeling and dual graph driven network (UPDGD-Net), which can fully leverage the supervised information of the available labels and feature information of available views. Different from the existing works, we leverage the label matrix to impose dual graph constraints on the embedded features of both view-level and label-level, which enables the method to maintain the inherent structure of the real data during the feature extraction stage. Furthermore, our network incorporates an uncertainty-aware pseudo-labeling strategy to fill the missing labels, which not only addresses the learning issue of incomplete multi-labels but also enables the method to explore more supervised information to guide the network training. Extensive experiments on five datasets demonstrate that our method outperforms other state-of-the-art methods.

Main paper: 
 introduction: 
 With the development of data acquisition technology, many researchers have found that multi-view data integrated from different sources can provide a more nuanced and diverse representation of an object. For example, an image can be described by different feature descriptors such as SIFT, Gist, and HSV. As a result, multiview learning has emerged as an important approach in the realm of data analysis, and many related works are based on subspace learning [21] [23] [26] and matrix factorization [13] [25] have been proposed. On the other hand, as a classical classification problem, multi-label classification (MLC) has also gaining prominence in the field of pattern recognition for a long time. Unlike single-label data which requires mutual exclusivity among labels, multi-label data contains various category tags and naturally maintains complex label correlations. For instance, an image might be described by multiple elements such as "neon signs", "sidewalks", and "passing cars", each contributing to a more comprehensive understanding of the scene. However, traditional multi-label classification methods often suffer from a significant issue as they primarily depend on features extracted from a single perspective, which limits their performance. By integrating multi-view learning into multi-label classification, this limitation can be effectively mitigated. As a result, a composite multi-view multi-label classification (MvMLC) has emerged and attracted increasing attention from researchers.MVMLC synergistically combines the strengths of both multiview learning and multi-label classification, offering a more robust and comprehensive framework for understanding complex multi-view multi-label data and plenty of related works have been proposed in recent years. For example, Sun and Zong proposed a Latent Conditional Bernoulli Mixture model called LCBM, which leverages a latent conditional Bernoulli mixture approach to integrate features from multiple views and using a shared latent subspace for label dependency construction [18] . Zhang et al. proposed a Latent Semantic Aware Multi-view Multi-label Learning approach, which utilizes matrix factorization and Hilbert-Schmidt Independence Criterion in kernel spaces to align latent semantic components across multiple views [27] . However, a significant challenge in MvMLC is the frequent occurrence of incomplete data, both in terms of views and labels. This challenge directly leads to the necessity of Incomplete Multi-View Multi-Label Classification (iMvMLC), which focuses on scenarios where some views or labels are missing. For instance, Tan et al. proposed the iMVWL, which attempts to learn a shared subspace that integrates weak label information and local label correlations [19] . Li et al. proposed an iMvMLC method based on matrix factorization, named NAIM3L, which bridges non-aligned views through common labels and exploits both global and local structural relations within multiple labels [9] . More recently, DNN-based methods have been introduced, offering improved performance by leveraging high-level feature extraction and complex model architectures, such as LMVCAT [12] and DICNet [11] . Despite these advances, existing methods struggle to fully utilize supervision information to help the model extract more discriminative features due to the incompleteness problem.To bridge this gap, our research introduces a novel network called UPDGD-Net for the iMvMLC task. Different from the existing works, the proposed UPDGD-Net tries to impute missing labels with credible pseudo-labels through an uncertainty-aware pseudolabeling strategy and then utilizes the filled label matrix to impose dual graph constraints on high-dimensional features extracted by the model. Specifically, we first design two distinct transformerbased modules, one dedicated to cross-view aggregation and the other for multi-label classification. Additionally, we apply average view token (AVT) in the former to better learn the consistent information across multiple views. Furthermore, our network harnesses the semantic guidance offered by the label matrix to largely maintain the intrinsic relationship of the original data by constraining the embedded features of both view-level and label-level. Last but not least, inspired by the self-paced learning [14] and uncertainty analysis from the Monte-Carlo Dropout [3] , we propose a novel uncertainty-aware pseudo-labeling strategy that maximizes the use of available data while maintaining the integrity and richness of the multi-label information. Overall, compared to existing methods, the contributions of our method can be summarized as follows:• To address the challenge of incomplete data learning, we propose the UPGDG-Net that incorporates information contained within missing labels through a novel uncertaintyaware pseudo-labeling strategy. UPDGD-Net attempts to maximize the use of available data while maintaining the integrity and richness of the multi-label information, thus enhancing the accuracy of the classification. • We introduce a label-guided dual-constraints approach in our network, which capitalizes on the untapped potential of multi-label semantics to preserve the intrinsic relationship of the original data in the embedding space. • We apply average view token (AVT) in our transformersbased architecture to better learn and synthesize the consistent information across multiple views. Extensive experiment results on five datasets confirm the effectiveness of our method.  
incomplete multi-view multi-label classification: 
 Previous studies in multi-view multi-label classification (MvMLC) have primarily relied on the completeness assumption of multiview multi-label data, which neglects the incompleteness scenarios in real-world applications. As a result, the incomplete multi-view multi-label classification (iMvMLC) task has received attention from plenty of researchers and corresponding methods need to consider the incompleteness issue in multi-view multi-label data. For instance, Tan et al. proposed iMvML, which learns a shared subspace that includes weak label information and local label correlations to manage cross-view relationships and incomplete label scenarios effectively [19] . Li and Chen introduced NAIM3L, focusing on both the global high rankness and the local low rankness in the label embedding space [9] . Zhu et al. proposed a method named WCC-MVML-ID, which integrates within-view, cross-view, and consensus-view representations to effectively process incomplete multi-view multi-label datasets. [30] . In addition to these traditional methods, DNN-based frameworks in iMvMLC have also shown significant performance. For example, Liu et al. proposed the deep iMvPMLC framework DICNet, which utilizes stacked autoencoders for view-specific feature extraction and introduces an incomplete instance-level contrastive learning scheme [11] . LMVCAT leverages the self-attention mechanism to extract high-level features and exploit inter-class correlations to enhance classification performance [12] .Assume that we are given a multi-view data 𝑋 and 𝑌 ∈ {0, 1} 𝑛×𝑐 is initialized as '0' in the data-preparation stage. The target of our method is to train a multi-label classification network on incomplete multi-view partial multi-label data, which can accurately predict the categories of unlabeled incomplete multi-view data. 
the proposed method: 
 In this section, we will introduce each component of the proposed UPDGD-Net, including a multi-view semantic representation learning framework, dynamically weighted fusion module, multi-label interaction module, label-guided dual constraints, and uncertaintyaware pseudo-labeling module. Fig. 1 shows the framework of our UPDGD-Net and Fig. 2 shows the process of our uncertainty-aware pseudo-labeling strategy.(1)(1) ℓ ℓ Classifier(2) ... As is well-known, the success of multi-view learning lies in exploiting the consistency and complementary information of multi-view data. To achieve this, similar to [12] , we design a new cross-view transformer encoder (CVEncoder) to facilitate the cross-view interaction and extract high-quality cross-view features. To ensure a uniform feature dimension across various views, we first employ Multilayer Perceptrons (MLP) to map the original multi-view data into a shared embedding space denoted as {Φ (𝑣) :𝑋 (𝑣) ∈ R 𝑛×𝑑 𝑣 → X (𝑣) ∈ R 𝑛×𝑑 𝑒 } 𝑚 𝑣=1, where 𝑑 𝑒 denotes the mapped dimension of multi-view data. Additionally, to ensure the consistency of multi-view features, we introduce some identical Average View Tokens (AVT) calculated by the average of all available views of each sample as extra inputs for CVEncoder. The calculation of the AVT corresponding to the 𝑖-th sample is formulated as follows:EQUATIONwhere Λ 𝑖 ∈ R 1×𝑑 𝑒 represents the AVT for the 𝑖-th sample. In (1), the missing view indicator matrix 𝑊 ∈ R 𝑛×𝑚 is introduced to disregard the missing views in each sample. Since the self-attention mechanism dynamically adjusts token influence based on the distribution of attention scores, allowing multiple identical tokens can amplify their impact within the sequence. In our method, we adopt B AVTs with the same value for each sample. Considering these tokens, the input of CVEncoder for the 𝑖-th sample can be expressed as:𝑉 𝑖 = Concat X𝑖 ; Λ 1 𝑖 ; ...; Λ B 𝑖 where 𝑉 𝑖 ∈ R (𝑚+ B) ×𝑑 𝑒 and Λ 1 𝑖 = ... = Λ B 𝑖 .Specifically, to mitigate the negative impact of missing views when calculating attention scores in the CVEncoder, we also utilize the missing view indicator matrix 𝑊 to mask the missing views during the computation of multi-head self-attention scores. The process for our masked multi-head self-attention layer , where ℎ represents the number of attention heads. To effectively mask the embedding features by the missing-view distribution, we expand the mask matrix 𝑊 to W ∈ R 𝑛× (𝑚+ B) by adding B column vectors with all one. Then we construct a mask matrix of sample 𝑖: 𝑀 𝑖 = w𝑇 𝑖 w𝑖 ∈ R (𝑚+ B) × (𝑚+ B) , where w𝑖 is 𝑖-th row vector of W . Then for 𝑖-th sample's input 𝑉 𝑖 , the view correlations 𝐴 𝑡 𝑖 and output 𝐻 𝑡 𝑖 is calculated as follows:EQUATIONEQUATIONwhere 𝑑 ℎ = 𝑑 𝑒 /ℎ represents the dimensionality per attention head and 𝐻 𝑡 𝑖 ∈ R (𝑚+ B) ×𝑑 ℎ is the output of 𝑡-th attention head. 𝑃 𝑞 𝑡 ∈ R 𝑑 𝑒 ×𝑑 ℎ , 𝑃 𝑘 𝑡 ∈ R 𝑑 𝑒 ×𝑑 ℎ , and 𝑃 𝑣 𝑡 ∈ R 𝑑 𝑒 ×𝑑 ℎ are projective matrices. Additionally, we introduce a fill function 𝑓 𝑖𝑙𝑙 (𝐴, 𝐵) before the softmax active function in the CVEncoder to set 𝐴 𝑖,𝑗 = -1𝑒 9 if 𝐵 𝑖,𝑗 = 0. With such a fill operation, the softmax can ignore the negative affect of missing views when calculating the attention scores. Finally, we obtain the output of CVEncoder for sample 𝑖 by concatenating the outputs of ℎ attention heads: Ṽ𝑖 = 𝐶𝑜𝑛𝑐𝑎𝑡 (𝐻 1 𝑖 , ..., 𝐻 ℎ 𝑖 ) ∈ R (𝑚+𝐵) ×𝑑 𝑒 . Overall, our CVEncoder can be simply formulated as:Γ : 𝑉 𝑖 ∈ R (𝑚+ B) ×𝑑 𝑒 𝑛 𝑖=1 → Ṽ𝑖 ∈ R (𝑚+ B) ×𝑑 𝑒 𝑛 𝑖=1and the first𝑚 rows of CVEncoder's output Ṽ𝑖 ∈ R (𝑚+ B) ×𝑑 𝑒 𝑛 𝑖=1 , i.e., {𝑍 𝑖 ∈ R 𝑚×𝑑 𝑒 } 𝑛𝑖=1 is regarded as the embedding feature of each sample.After obtaining the embedding features of each view using CVEncoder, our goal is to create a unified and consistent representation that fully characterizes the multi-view data. However, different views may contribute differently to this representation. To account for varying discriminative abilities among multiple views, we adopt a dynamically weighted fusion approach inspired by [12] . We define the fused feature 𝑓 𝑖 for sample 𝑖 as follows:EQUATIONwhere 𝑧 (𝑣) 𝑖 denotes the embedding feature of the 𝑣-th view/row in 𝑍 𝑖 and 𝑎 𝑣 represents a learnable scalar weight for the 𝑣-th view and 𝑎 𝑣 is updated by network. Notably, to mitigate the negative impact of missing views during multi-view fusion, we introduce a missingview indicator matrix 𝑊 into our fusion module. This equation captures an adaptive weighting mechanism for each view, resulting in a comprehensive feature representation that incorporates information from all available views.When we obtain a consistent common representation from incomplete multi-view data, another crucial issue is how to utilize the relationships between multiple labels to enhance the discriminative power of the multi-view representation. To achieve this purpose, inspired by [7] . we introduce label embeddings to map each category directly into the feature space, leveraging the self-attention mechanism to capture category correlations. More specifically, we randomly initialize 𝑐 label embedding features as 𝑐 class tokens is regarded as the final discriminative feature for classification. Finally, our CCEncoder can be simply formulated as:𝑐𝑙𝑠 𝑖 ∈ R 𝑑 𝑒Υ : 𝐶 𝑖 ∈ R (𝑐+1) ×𝑑 𝑒 𝑛 𝑖=1 → C𝑖 ∈ R (𝑐+1) ×𝑑 𝑒 𝑛 𝑖=1In our work, we attempt to introduce two label-guided constraints to enhance the quality of the latent representation and the initialized class tokens.Label-guided constraint on latent representations: Generally speaking, original data typically exhibit a natural topological structure, where a part of samples (such as nearest neighbor samples) or categories share similarities while differing from others. For example, some samples may simultaneously annotated with two or more identical labels. Unfortunately, most deep learning methods overlook this structure when transforming data into feature space. To address the above issue and obtain a more reasonable representation yet good classification performance, following [12] , we introduce a label-guided constraint on latent representations𝑍 𝑖 ∈ R 𝑚×𝑑 𝑒 𝑛 𝑖=1that leverages the rich semantic information of multiple categories to guide the network training. Firstly, based on the smoothness assumption that if two samples have more identical labels, they should also have more similarity [24] , we construct a label-based correlation graph 𝑄 ∈ R 𝑛×𝑛 according to the available label data as follows:EQUATIONwhere ⊙ denotes Hadamard product. 𝐺𝐺 𝑇 is used for normalization in the calculation of the label-based correlation matrix 𝑄. This equation quantifies the similarity between samples based on the number of common positive labels they share. In simpler terms, two samples are considered more similar if they share a larger number of common positive tags. To this end, we further calculate the similarity between two embedding features within each view using cosine similarity:EQUATIONwhere 𝑧 Label-guided constraint on class tokens: In our network, we introduce 𝑐 class tokens as input of the CCencoder to improve the interaction of the multi-view features and labels. Correspondingly, we obtain 𝑐 label embedding features, which can be regarded as the discriminative representation of 𝑐 categories. For these label embeddings, we also expect them to preserve similar label correlations as the original labels. To this end, similar to [5] , we introduce a label-guided constraint on these label embeddings. Firstly, we construct a new label correlation graph ℑ ∈ R 𝑐 ×𝑐 in which each element represents the probability of a sample having both the 𝑖-th and 𝑗-th labels simultaneously as follows:EQUATIONwhere 𝑛 𝑘=1 𝑌 𝑘,𝑖 𝑌 𝑘,𝑗 calculates the total number of co-occurrences of category 𝑖 and category 𝑗. 𝑛 𝑘=1 (𝑌 𝑘,𝑖 + 𝑌 𝑘,𝑗 ) refer to the sum of the number of occurrences of category 𝑖 and category 𝑗.Similarly, for the label embedding features corresponding to the outputted class tokens from CCEncoder, we can calculate their cosine distance-based similarities as follows:EQUATIONwhere c𝑙𝑠 𝑖 denotes the label embedding of 𝑖-th category. Then, to preserve the similar structure as labels, we introduce the following loss on the label embeddings:ℓ 𝑔𝑐𝑙 = 1 𝑐 (𝑐 -1) 𝑐 ∑︁ 𝑖=1 𝑐 ∑︁ 𝑗≠𝑖 𝐾 𝑖,𝑗 -ℑ 𝑖,𝑗 2 (10)Pseudo-label strategy has proven its effectiveness in semi-supervised learning and unsupervised learning tasks. For the incomplete label learning network, we also design an uncertainty-aware pseudolabeling strategy to explore more supervised information of the incomplete label data. In the designation of our pseudo-labeling strategy, the key challenges are evaluating the uncertainty of the predicted labels and selecting the credible pseudo-label to guide the model training. Inspired by [3] , we apply the Monte-Carlo Dropout (MC dropout) approach to quantify the uncertainty of the predicted labels. Dropout is implemented by probabilistically deactivating neurons using a Bernoulli distribution, making the network's output 𝑦 * for an input 𝑥 * follow a predictive distribution rather than a definite value. is the binary vector for the 𝑖-th layer in the 𝑡-th dropout operation. Each neuron's dropout, 𝜗 𝑡 𝑖,𝑗 , is drawn from a Bernoulli distribution 𝜗 𝑡 𝑖,𝑗 ∼ 𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖 (𝑝), with 𝑝 equal to the dropout ratio used during training. Then we can obtain T groups of network parameter sets as follows: 11) where • denotes the Hadamard product.EQUATIONAs shown in Fig. 2 , given dataset 𝑋 with 𝑛 samples, we can perform stochastic forward propagation𝑇 times withΘ 𝑡 1 , Θ 𝑡 2 , ...Θ 𝑡 𝐿 T 𝑡 =1and obtain T sets of outputs Ŷ 1 , ..., Ŷ T , where Ŷ t ∈ R 𝑛×𝑐 . We can estimate the mean of the outputs as follows:EQUATIONwhere 𝑞 Ŷ |𝑋 represents the approximate predictive distribution after T sampling times. Then we estimate:EQUATIONFinally, we can calculate the variance of the model's output:EQUATIONThen we can normalize the variance to obtain the uncertainty of the output:EQUATIONwhere 𝑈 𝑖,𝑗 denotes the uncertainty of the 𝑗-th output of the 𝑖-th sample and 𝑈 ∈ [0, 1] 𝑛×𝑐 . After that, we can treat the predictive mean of T outputs E 𝑞 Ŷ |𝑋 Ŷ as candidate pseudo-labels and select credible pseudo-labels from it based on uncertainty. Inspired by [15] , we apply a self-paced selection strategy to select these reliable pseudo-labels. Firstly, we introduce an age parameter 𝜆 and initialize its value as 0 and progressively increment its value as follows:EQUATION)where 𝑒 denotes the 𝑒-th epoch and ê denotes the overall training epochs. Specifically, we introduce a temperature coefficient 𝜏 to control the growth rate of 𝜆. Gradually increasing 𝜆 allows for the selection of more credible candidate pseudo-labels over time. Then we construct weight matrix 𝐷 ∈ R 𝑛×𝑐 as follows:EQUATION)where 𝜆 is a gradually increasing dynamic threshold and we mark output positions with uncertainty less than this threshold as 1 on 𝐷 for subsequent selection. To select pseudo-labels for the missing label, we need to adjust weight matrix 𝐷 based on missing-label indicator 𝐺: D = 𝐷 ⊙ G where G𝑖,𝑗 = 1-𝐺 𝑖,𝑗 indicates the position of missing labels in label matrix. Then we can select soft pseudo-labels from candidate pseudo-labels for the next epoch:EQUATIONFinally, we can obtain the final output 𝑂 = Ψ f ∈ [0, 1] 𝑛×𝑐 , where Ψ is the final classifier. Then we compute the following multi-label classification loss using a masked binary cross-entropy function, which incorporates both observed labels and pseudolabels as follows:EQUATIONwhere 𝐺 is introduced to calculate the loss from observed labels in the first term and Ỹ in the second term represents the soft pseudolabels selected from the previous training round. Overall, our total function will be:EQUATIONwhere 𝛼 and 𝛽 are penalty coefficients. The training process is described in Algorithm 1 
experimental setup: 
 Dataset Description and Preparation: Following [9, 11, 19] , we train and validate our model on five well-known multi-view multilabel datasets: Corel5k [1] , Pascal07 [2] , ESPGame [22] , IAPRTC12 [4] , and MIRFLICKR [6] . Each dataset includes six views represented by GIST, HSV, Hue, Sift, RGB, and LAB features extracted from their respective datasets. To simulate real-world scenarios with missing views and partial labels, we construct incomplete multi-view partial multi-label data as follows: (1) We randomly mask 50% of samples for each view, ensuring at least one available view per sample. (2) For each category, 50% of both positive and negative tags are randomly removed. Subsequently, 70% of these samples are randomly chosen as the training set. Method Comparison: We evaluated our method's effectiveness by comparing it with six well-known methods: iMvWL [19] , NAIM3L [9] , CDMM [29] , DeepIMV [8] , DICNet [11] , and LMVCAT [12] . iMvWL and NAIM3L are non-DNN iMvMLC models, while the rest are DNN-based. Notably, CDMM and DeepIMV, which are for 𝑒=0 to ê do for v=1 to m do Compute X (𝑣) = Φ (𝑣) 𝑋 (𝑣) end for Compute AVT using Eq.( 1). Compute Ṽ = Γ (𝑉 ). Extract embedding feature 𝑍 from Ĩ . Compute ℓ 𝑔𝑐𝑠 using Eq.( 7). Compute 𝑓 using Eq.( 4). Compute C = Υ (𝐶). Compute ℓ 𝑔𝑐𝑙 using Eq. (10) . Extract discriminative feature f from H . Compute 𝑂 = Ψ( f ). Compute 𝑈 using Eq.( 15). Obtain pseudo-labels Ỹ by Eq.( 17). Compute ℓ 𝑐𝑒 using Eq.( 18). Compute ℓ 𝑡𝑜𝑡𝑎𝑙 using Eq. (19) .Update Φ (𝑣) 𝑚 𝑣=1 , Γ, Υ, {𝑐𝑙𝑠 1 , ...𝑐𝑙𝑠 𝑐 }, Ψ.end for Output: Trained model parameters MvMLC methods, cannot handle missing data natively. To implement the two methods on the incomplete multi-view multi-label classification tasks, we used mean imputation for missing views and set '0' for unknown tags. DICNet is a contrastive learning framework with InfoNCE loss, and LMVCAT is a transformer-based method. The above two methods can handle incomplete multi-view and partial multi-label data. Parameters for these methods were set as per recommendations in their original publications for fairness.In this section, we evaluate the performance of our method for the classification task by comparing it with six state-of-the-art algorithms on the five datasets. Table 1 shows the experiment results of the five evaluation metrics, in which the missing-view rate and missing-label rate are both arranged as 50%. Fig 3 shows more results related to different missing views and missing label ratios on the Corel5k dataset to investigate the impact of missing data. According to the statistical results in Table 1 , it is evident to make the following observations:• On all metrics of both five datasets, our method achieves superior performance compared with other methods, which fully demonstrates the effectiveness of our method on the iMvMLC task. • Compared to traditional methods, the methods that are specifically designed for double incompleteness show obvious advantages compared to other methods. This indicates the compared to conventional pseudo-labeling strategy. This observation illustrates the superiority of our uncertainty-aware pseudolabeling strategy in identifying and selecting the most credible pseudo-labels for our model, further reinforcing its effectiveness. On B and T : From Fig 4c, it can be seen that too large or too small of the number of average view tokens is not conducive to achieving optimal performance. Based on experiments, we set B as 3 for all datasets. Fig 4d shows the AP value versus hyper-parameters T , which can be seen that when T is greater than 4, our method is not sensitive to it, so we set T = 5 for all five datasets.The ablation experiments are conducted on the Corel5k and ESPGame datasets, with 50% missing views, 50% missing labels, and 70% training samples. We sequentially removed AVT, ℓ 𝑔𝑐𝑠 , ℓ 𝑔𝑐𝑙 , and the uncertainty-aware pseudo-labeling strategy (PLS). Notably, we leverage pseudo-labels selected from PSL as extra supervised information to calculate the classification loss in our network. Consequently, we remove the cross-entropy loss between the predictions and pseudo-labels to validate the effectiveness of our PSL. The results presented in Table 2 , reflect two key observations: (i) Each component of our UPDGD-Net is crucial and contributes positively, emphasizing the significance of every component in enhancing multi-label classification performance. (ii) The most significant enhancement is observed with the integration of the SPL. 
conclusion: 
 In this paper, we propose an uncertainty-aware pseudo-labeling and dual graph driven network (UPDGD-Net) for iMvMLC tasks, designing an uncertainty-aware pseudo-labeling strategy to harness the semantic information in missing labels. We also utilize multi-label topological relationships to guide representation learning in embedding space and propose the average view token to enhance the multi-view information synthesis. Extensive experiments validate the effectiveness of our approach.  
 
 Appendix:  


Decision: This paper received wa, wa, ba, br from reviewers after rebuttal. Reviewers believe that this paper is the first work to learn discriminative topological information under incomplete scenarios. There are also some concerns from reviewers such as motivation and related works. Authors should solve these issues in their final version.

Reviews: review: paper_summary: paper_summary: The paper introduces the UPDGD-Net framework named of uncertainty-aware pseudo-labeling and dual graph driven network, which focuses on a practical problem for incomplete multi-view learning. It tackles two main challenges: How to obtain an effective representation with incomplete views and How to choose credible pseudo-labels for missing labels. Compared with the existing methods, the authors design dual graph constraints on the embedded features of both view-level and label-level. The experimental results on several real-world datasets demonstrate the effectiveness of the proposed model.
strength_weakness: strength_weakness: 1. The paper is well-motivated since missing is a common and significant problem in real-world multi-view learning. The model design for the missing problem in this paper is very reasonable, offering a highly comprehensive coverage.

2. The paper provides a well-structured overview, making it easy to follow for readers. Besides, the implementation details of the selected technique are described in detail.

3. The experiments were conducted in several benchmark datasets, showing the superiority of the proposed method.
limitations: limitations: 1. About novelty: Beyond the framework innovation, there seems to be absence of original techniques and methods, with the approach primarily consisting of an amalgamation of existing techniques. Besides, the runnable code is not provided, making readers hard for reproduction of Label-Guided Dual Constraints and Pseudo-Labeling module.

2. About clarity: Lack analysis about how previous researches addressed the problem about tackling Pseudo-Labeling for missing labels. Besides, why uncertainty-based pseudo-labeling can be seen as the credible pseudo-labels? How to define credible pseudo-labels? The authors should give theoretical proof or discuss and analyze.

3. About soundness: Lack of time analysis and how is the complexity of this method compared to the comparison method? It seems that the proposed algorithm has a high time complexity.

4. About experiments: From my point of view, the setting of the experiments is unfair and biased. Since the proposed UPDGD-Net compared with the comparison methods has both feature and label completion modules, the experimental settings should be conducted under different missing rates, like [1]. Why only consider the case that the missing rate is 50%? How about the performance under missing rate from 60% to 90%. It seems that the effectiveness of the model can be better reflected in the case of extreme missing (90% missing rate).

[1] Lin, Zhenghong, et al. "Contrastive Intra-and Inter-Modality Generation for Enhancing Incomplete Multimedia Recommendation." Proceedings of the 31st ACM International Conference on Multimedia. 2023.
paper_summary: paper_summary: This paper considers multi-view multi-label classification (MvMLC) problem with missing labels and views of data and proposes a graph-based framework UPDGD-Net. The proposed UPDGD-Net models and aligns data samples by constructing a cross-view and a cross-label graph, as well as a pseudo-labeling strategy to enhance model performance. Comprehensive experiments illustrate the performance of the proposed method.
strength_weakness: strength_weakness: 1. The paper is well written and easy to follow.
2. A comprehensive architecture that models both samle and label correlation with a contrastive graph learning framework.
3. Experimental results demonstrate the superiority of the UPDGD-Net framework.
limitations: limitations: 1. As the authors have frankly mentioned, most of the designs in UPDGD-Net are originated from previous works on multi-view learning [1], multi-label learning [2], as well as other label-utlization techniques [3], which raise the concerns over the novelty of this work.
2. The motivation of using a Monte-Carlo Dropout strategy, as well as how the proposed UPDGD-Net would address MvMLC tasks with incomplete information require further explanations.
3. The performance of baseline models with complete label and view information should also be concluded to make a comprehensive comparison.




[1] Incomplete MultiView Multi-Label Learning via Label-Guided Masked View- and Category-Aware Transformers, AAAI 2023

[2] General Multi-label Image Classification with Transformer, CVPR 2021

[3] Collaborative Learning of Label Semantics and Deep Label-Specific Features for Multi-Label Classification, TPAMI 2022
paper_summary: paper_summary: This paper introduces UPDGD-Net, a framework designed to handle multi-view multi-label learning with imcomplete view/label. This network features an uncertainty-aware pseudo-labeling strategy to fill missing labels with pseudo-labels, achieved by Monte-Carlo Dropout to quantify uncertainty. It also imposes dual graph constraints on embedded features at both the view and label levels to maintain the inherent data structure. Additionally, transformer-based modules are used for cross-view aggregation and multi-label classification, utilizing average view tokens (AVT) to ensure consistency. Experiments on five datasets demonstrate that UPDGD-Net outperforms existing methods, effectively addressing incomplete data issues and improving classification accuracy.
strength_weakness: strength_weakness: 1. This paper studies an interesting yet understudied area, which is multi-view multi-label learning with incomplete views and labels.

2. This paper is overall well-written and well-structured.

3. The proposed modules undergo ablation studies and suggest the effectiveness of the individual modules.
limitations: limitations: Major issues:

1. This paper proposed a lot of different modules - yet the proposition of those modules is not well-motivated. I do not see strong associations between those modules and the problem of missing view/label. 

2. Some relevant works [1,2] are omitted - is there a reason why those works are not included or compared? More importantly, the problem of both missing view and missing label has been studied by [2], but there is no discussion on the limitations and differences between the proposed method and [2] whatsoever.

3. The proposed uncertainty-aware pseudo-labeling straightforwardly applies the Mote Carlo dropout, which does not appear to be novel, in my opinion.

Minor issues:

1. The notations and problem definition should not be included in related works.

[1] Multi-View Multi-Label Learning with View-Specific Information Extraction, IJCAI 2019.

[2] Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views, TNNLS 2023.
paper_summary: paper_summary: The paper proposes a novel approach for handling incomplete multi-view multi-label classification (iMvMLC) tasks. This is particularly relevant in fields such as medical imaging and bioinformatics, where missing data are common due to various uncertainties in data collection and labeling. Key novelties include Uncertainty-Aware Pseudo-Labeling (uses a strategy that employs uncertainty estimation to generate pseudo-labels for missing labels, enhancing the learning process despite incomplete data)  and Dual Graph Constraints (leverages a label matrix to apply constraints on feature embeddings at both the view and label levels, helping maintain the intrinsic structure of the data during feature extraction). The model uses transformer encoders to learn multi-view representations and label interactions. It outperforms 6 previous methods on 5 benchmark datasets in experiments. Ablation studies validate the importance of the graph constraints and pseudo-labeling components.
strength_weakness: strength_weakness: (1) The paper introduces an innovative approach to handling incomplete multi-view and multi-label data through uncertainty-aware pseudo-labeling and dual graph constraints. This method is novel as it leverages the rich semantic information in the label space to preserve the intrinsic data structure during representation learning, and it augments the incomplete label information by selecting credible pseudo-labels based on prediction uncertainty.  
(2) The paper provides comprehensive experimental results to validate the effectiveness of the proposed methods. Testing the model on five different datasets and comparing it with several SOTA methods, demonstrates its practical utility and robustness.  
(3) One of the key strengths of the paper is its focus on robustness to incomplete data, which is a common issue in practical applications. The methodology developed is particularly relevant for areas like medical imaging and bioinformatics, where missing data are frequent.
limitations: limitations: (1) The proposed UPDGD-Net has a complex architecture involving multiple transformer encoders, graph constraints, and pseudo-labeling. The high complexity makes it difficult to interpret and analyze the model's behavior and the contributions of individual components. Iterative uncertainty estimation via Monte Carlo dropout sampling and the transformer architectures can be computationally expensive, especially for large datasets. The paper does not discuss the model's efficiency and costs.  
(2) I feel like there are many hyperparameters in these proposed methods. Although some hyperparameters are discussed in Section 4.3, others remain undiscussed. For example, the pseudo-labeling strategy involves multiple hyperparameters (e.g., temperature coefficient, growth rate, uncertainty threshold) whose selection criteria and sensitivity analysis could be better explained.  
(3) The results of Figure 3 could be more thoroughly explained. Since there are many evaluation metrics, the authors could first state which one is the primary metric. Then, they could provide more details about the figure. For example, they could explain why the OE and Cov metrics increase as the missing-view ratio and missing-label ratio continue to grow.