Learning Compact Semantic Information for Incomplete Multi-View Missing Multi-Label Classification
Authors: Jie Wen
Yadong Liu
Zhanyan Tang
Yuting He
Yulong Chen
Mu Li
Chengliang Liu
Venue: ICML 2025 poster (2025)
Keywords: Multi-view multi-label classification
Incomplete multi-view learning
Missing multi-label classification
Primary area: general_machine_learning->supervised_learning
TL;DR: We propose a novel cross-view shared information learning framework for incomplete multi-view missing multi-label classification.

Abstract: Multi-view data involves various data forms, such as multi-feature, multi-sequence and multimodal data, providing rich semantic information for downstream tasks. The inherent challenge of incomplete multi-view missing multi-label learning lies in how to effectively utilize limited supervision and insufficient data to learn discriminative representation. Starting from the sufficiency of multi-view shared information for downstream tasks, we argue that the existing contrastive learning paradigms on missing multi-view data show limited consistency representation learning ability, leading to the bottleneck in extracting multi-view shared information. In response, we propose to minimize task-independent redundant information by pursuing the maximization of cross-view mutual information. Additionally, to alleviate the hindrance caused by missing labels, we develop a dual-branch soft pseudo-label cross-imputation strategy to improve classification performance. Extensive experiments on multiple benchmarks validate our advantages and demonstrate strong compatibility with both missing and complete data.

Learning Compact Semantic Information for Incomplete Multi-View Missing Multi-Label Classification
Jie Wen! Yadong Liu!
Zhanyan Tang'
Yuting He? Yulong Chen
Mu Li!
Chengliang Liu'
Abstract
et al., 2023), and multimedia information retrieval (Fang et al., 2022). Not only does multimodal data with significant differences exhibit rich representational power, but multi- feature data from the same sample also holds substantial representational capabilities. The learning paradigm for performing machine learning or data analysis tasks based on such multimodal or theoretical framework of multi-view learning (Zhang et al., 2019). And the success of this framework in downstream tasks is largely attributed to extracting and leveraging multi- view consistent semantic information (Hwang et al., 2021; Zeng et al., 2023; Fang et al., 2023).
Multi-view data involves various data forms, such as multi-feature, multi-sequence and multimodal data, providing rich semantic information for downstream tasks. The inherent challenge of in- complete multi-view missing multi-label learning lies how to effectively utilize limited supervi- sion and insufficient data learn discriminative representation. Starting from the sufficiency of multi-view shared information for downstream tasks, we argue that the existing contrastive learn- ing paradigms on missing multi-view data show limited consistency representation learning ability, leading to the bottleneck in extracting multi-view shared information. In response, we propose to minimize task-independent redundant informa- tion by pursuing the maximization of cross-view mutual information. Additionally, to alleviate the hindrance caused by missing labels, we develop a dual-branch soft pseudo-label cross-imputation strategy to improve classification performance. Extensive experiments on multiple benchmarks validate our advantages and demonstrate strong compatibility with both missing and complete data.
data is unified within the
In traditional single-label classification tasks, it is typically assumed that each sample corresponds to a unique label. However, in real-world applications, almost all samples may be associated with multiple labels. For example, in autonomous driving tasks, images captured by cameras may contain multiple entities and be associated with several la- bels. Multi- Learning (MLL) tasks have attracted increasing attention due to their strong practical significance (Zhou et al., 2012), which aims to predict all possible rele- vant labels for unlabeled example. Recent extensive re- search has witnessed significant progress of MLL in various practical applications, e.g., document classification (Fallah et al., 2023), emotion recognition (Zhang et al., 2021; Ameer et al., 2022), and image annotation (Chen et al., 2019).
1. Introduction
Combining the characteristics of the two tasks mentioned above, multi-view multi-label classification (MVMLC) methods (Wu al., 2019; Zhang al., 2020) aims to leverage multi-view semantic information of samples as much as possible to enhance the performance of multi-label classification. However, in real-world data collection pro- cesses, failures often occur inevitably due to various rea- sons. For example, in medical diagnosis, missing diagnostic reports often occur due to privacy concerns or economic factors. Additionally, manual annotation of labels may lead to incomplete tagging. This leads to the necessity of in- complete Multi-view Missing Multi-Label Classification (iM3C), which focuses on scenarios where some views and labels are missing.
Multimodal learning aims to integrate data from different modalities (such as images, text, speech, etc.) to enhance the model's understanding and decision maaiing capabilities. This approach has shown significant potential in fields such as medical diagnosis (Qiu et al., 2022; Luo et al., 2023; Lin et al., 2025), wearable devices (Wen et al., 2022: Fang et al., 2024), sentiment analysis (Soleymani et al., 2017: Zhang
'School of Computer Science and Technology, Harbin Insti- tute of Technology, Shenzhen, 518000 China DDepartment of Biomedical Engineering, Case Western Reserve University, Cleve- land, OH 44106 USA. Correspondence to: Chengliang Liu <li- uc11996@ 163.com.>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Considering the incomplete multi-view problem, Zhao et al. proposed a method named CDMM (Zhao al., 2021).

---

COME: Incomplete Multi- View Missing Multi-Label Classification
which explores multi-view consistency and diversity while employing an ensemble learning strategy to obtain the final results. To alleviate insufficient available label information, Zhao et al. proposed a method named LVSL based on the structural consistency between the view and the label space (Zhao et al., 2022). Li et al. approached the problem from the perspective of incomplete multi-label learning, leverag- ing the prediction matrix to capture global and local label structures to address the issue of incomplete labels (Ma & Chen, 2021). The above three methods have made signif- icant contributions to the fields of multi-view learning or multi-label classification. However, they are unable to fully address the iM3C problem. Therefore, Tan et al. proposed a method named iMWWL, which can simultaneously learn shared subspace from incomplete views with missing la- bels (Tan al., 2018). Li al. proposed method named NAIM3L, which constructs composite index matrix involy- ing available views and known label information to mitigate negative impacts (Li & Chen, 2021). In recent years, deep learning has achieved significant development (Yang et al., 2023b). In addition to above traditional methods, Liu et al. introduce deep incomplete instance-level contrastive learn- ing in the iM3C task to extract multi-view features (Liu et al., 2023). To further leverage supervised information, Xie et al. applied dual graph constraints to view-level and lebel-level embeddings, to generate missing labels through uncertainty-aware strategies (Xie et al., 2024b). Consider- ing the coupling between multi-view consistency and multi- label classification tasks, a method named SIP is proposed to construct label prototypes in a data-driven manner, which further extracts task-relevant information via the informa- tion bottleneck approach (Liu et al., 2024).
negative impact of missing supervisory information.
Overall, for iM3C task, we propose novel consistent semantic representation learning framework, named as COME. Compared to existing methods, our contributions can be summarized as follows:
We propose a multi-view semantic consistency en- hancement strategy to learn mulpact multi-view shared information, which effectively alleviates the perfor- mance degradation caused by incomplete contrastive learning.
Compared to existing works, we adopt dual-branch co-training approach, dynamically generating soft pseudo-labels for missing multi-label imputation.
Our COME shows good adaptability to arbitrary view and label incompleteness. Extensive experimental re- sults, both in complete and incomplete cases, demon- strate the superiority and robustness of our method.
2. Preliminary and Motivation
2.1. Problem Definition
For the iM3C task, we define variable set (x(1) )em any sample with observations, where ar(")) E Rd is i-th sampling of variable x(") and de denotes the dimensionality of e-th view. And we define y E {0, 1 as the incomplete label set, where c is the number of categories. Taking into account the impact of missing views and missing labels, we define V. with [V] S m, as the set of available views. Specifically, fx(*)) hevv (X' for short) denotes the views that can be observed. Similarly, we also define G, with | S to denote the set of known categories. We define n as the number of samples. Our goal is to learn the multi- view joint representation z from X which contains all the semantic consistency information that can be used to accurately predict the corresponding categories.
Je=1 as
The success of these methods is closely tied to the extrac- tion and learning of multi-view consistency information (i.e., shared information). To take a step further, we assume that almost all the information relevant to downstream clas- sification tasks is shared among views (Federici et al., 2020; Tsai et al., 2020). Despite the significant progress made by existing contrastive learning based methods in extracting consistent semantic features, we observe that the effect of contrastive learning in aggregating multi-view latent fea- tures remains limited on incomplete data, showing the inad- equacy of contrastive learning in dealing with incomplete multi-view data. Therefore, in this paper, we propose a mu- tual information enhancement strategy based on a Mixture of Experts (MoF) framework, which compresses the shared information in cross-view joint representations by maximiz- ing cross-view mutual information between shared represen- tations and the raw data. In addition, the insufficient of weak supervisory information in missing multi-lahel data also hin- ders the further improvement of multi-label classification performance. In response this, we adopt a dual-branch soft labudo-label cross- insssiimptttioon strategy to alleviate the
2.2. Sufficiency of Multi- Shared Representation
Suppose for each sample we just have two views, and z(), 22) are latent representations of x(1)x((2) and we introduce variable T to represent the classification-reateee information. Since we suppose all the relevval informa- tion contained by multi-view shared representation, inspired by (Federici et al., 2020; Tsai et al., 2020), we introduce the following redundancy assumption:
Assumption 2.1. (1) and x2) share the same semantic
information, which is sufficient for prediction: I (xx()), =
((x T). That is to say, if i!) satisfy I(x("); (2) (z () =
O, then (1) contains all the task-relevant information for prediction: T((x(1),x x(2)::) = T(x():)) = (T)
2

---

COME: Incomplete Multi- Missing Multi-Label Classification
In multi-view applications, the representations of all views are combined to fully leverage the shared semantic informa- tion. Furthermore, we introduce the semantic sufficiency proposition for the multi-view joint representation z:
the proportion of shared information in the representation. However, we observed that shared representations learned through contrastive learning on incomplete data are insuf- ficient. As shown in Fig. 1, we perform experiments on two popular multi-view multi-label datasets Corel5k and Pasca107 to illustrate the performance degradation of in- adequate contrastive learning. The ratio of missing views is set to 50% and the missing instances do not participate in the construction of positive-negative pairs in contrastive learning loss. We can observe that, compared to the case of complete views, the contrastive learning running on in- complete data leads to worse classification performance. As we know, the core idea of contrastive learning is pull to- gether the positive sample pairs and push apart the negative samples (Khosla et al., 2020). In the case of missing views, the contrastive learning less effective in pushing negative pairs away and pulling positive pairs closer. To address the inadequacy of contrastive learning under missing multi- view data, we propose a MoE-based mutual information enhancement framework, which aims to learn the minimal sufficient representation of multi-view data by compressing redundant information and maximizing cross-viee view mutual information.
Proposition 2.2. If z contains all the shared information among views, z. is sufficient for prediction.
From Proposition 2.2, we have [2; (xx(2) [(x(") T). It should be noted that if z contains all the in- formation from x() and x(2), it inherently includes all the information required for downstream tasks. However, in practical applications, the redundant information contained in will lead to performance degradation. Thus, we should compress the non-shared information within the joint rep- resentation as much as possible, i.e., pursuing the minimal sufficient shared representation cross all view:
Definition 2.3. (Minimal and Sufficient Shared Represen- tation) The joint representation z is minimal and sufficient for (2) and (2) if and only if [(x(2,, x x(2) x()) T(2, )2)
2.3. Insufficient Multi-view Contrastive Learning
Empirical evidence shows that self-supervised learning (SSL) methods, even without utilizing any downstream su- pervisory information during training, can still learn em- beddings that generalize well to a wide range of down- stream tasks. For example, SimCLR (Chen et al., 2020) defines contrastive loss and uses augmented images self-supervised signals, allowing the trained model to per- form effectively on various tasks. In previous frameworks, self-supervised signals were often generated through data augmentation techniques, such as word masking (Devlin, 2018) or image rotation (Gidaris et al., 2018). In multi-view contrastive learning, we consider that variables belonging to the same sample but from different views should share semantic consistency information, while variables from dif- ferent samples may not (Tian et al., 2020; Pan & Kang, 2021; Liu et al., 2023).
Average predston(AP)
0.42
0.55
0.553
8.38
$0.50
0.30
sufficient learning
sufficient contrastive learning
6o BO
100
0.45
15
20
65
60
Training Epochs
Training Epochs
(a) Corel5k
(b) Pasca107
Figure I. Imppct of contrastive loss on Average Precision (AP) in Corel5k and Pasca107 datasets under incomplete and complete multi-vien data.
2.4. Pseudo Label Learning
In the iM3C task, labels are also incomplete and previous methods leverage incomplete label indicator to mask the missing part in the training process. To reduce the negative impact of missing labels, inspired by some SSL. methods (Lee et al., 2013; Berthelot et al., 2019; Sohn et al., 2020; Xu et al., 2021), an intuitive strategy is to assign pseudo- labels to samples with missing labels. Pseudo-labeling is originally proposed for the semi-supervised training of deep neural networks (Lee et al., 2013). Some methods (Berthelot et al., 2019; Sohn et al., 2020) combine consistency regular- ization or entropy regularization to improve the quality of pseudo-labels. A common SSL method named Fixha (Sohn et al., 2020) aims to select the most probable label as the pseudo-label, known as T strategy. However, even without considering correctness, it is obvious that this method will neglect multiple true labels. In Fig. 2 (a), for the Pasca107 dataset, the pseudo-labels generated using the
Existing studies have demonstrated the positive correlation between contrastive learning and the maximization of inter- view mutual information (Tsai et al., 2020; Wang et al., 2022). That is to say, the objective of contrastive learn- ing is equal to max (xx 222)). From multi-view data {x()), (()))(()) x(2) and their latent representations 1z()),(())
based on the Data Processing Inequality (Thomas & Joy,
2006) and Markov chain, we have:
1(x(1) ((e)) 2 (1) 2 (12(11) (01))
(1)
The gap between ((x(1), x2) and (1) (2) will be x2)))
small enough when the model capacity is sufficient and the training data is sufficient. Ideally, contrastive learn- ing can compress non-shared information and maximize
3

---

COME: Incomplete Multi- View Missing Multi-Label Classification
Missing truth
For achieving the maximization in Eq. (2), we adopt the in- complete multi-view contrastive learning method proposed by (Liu et al., 2023), then we have following contrastive loss:
Top-2 2 Psesodo- Label
1200
0.4
0.3
600
LCTR = -E log exp([(z; (0) (!)
0
(3)
Category Index
categories sample
exp (1) 24 )20) Sneg
(a)
(b)
where Sneg = Er=v, E 2j=1,t exp(f(2) (0) (1)
.2 }), (v)
2j
Figure 2. (a) 1 pseudo-labels and missing ground-truth labels at Pascal07 dataset. (b) Label count distribution across samples in Pascal07 dataset.
means the sampling from distribution of z(1), and f is a function to measure the cosine similarity between sample- pairs. To do this, we adopt the view-specific stochastic encoder to approximate the distribution p(z(")/x(")) (x(")). Simi- lar to previous work (Shi et al., 2019; Wen et al., 2020), we also propose to factorise the joint variational posterior as a combination of unimodal posteriors, using a MoE to model p(z|X²), therefore we have:
above Top-1 generation strategy are less than the actual number of missing labels. An alternative choice to select the Top-k probable labels as pseudo-labels, but in practice it is difficult to set k. As shown in Fig. 2 (b), the number of true positive labels of samples in Pasca107 dataset is not constant number. The label count distribution across sam- ples for other datasets can be found in Appendix E. Based on this, the simple label generation strategy will inevitably introduce false positive labels or neglect true positive labels.
p(z|X²) Σ DEV
P (x()))
(4)
Furthermore, considering the class-imbalance gap between different classes (Yang et al., 2021; Fang et al., 2025), some studies propose to generate pseudo-label through com- plex methods, including the class-aware method (Xie et al., 2024a), the class-threshee based method (Xiao et al., 2024), and the label prototype-based method (Yang et al., 2023a).
3.2. Compact Shared Representation Learning
Reviewing the analysis in Section 2.3, we know that opti- mizing learning objective (2) with nulti-view contrastive learning alone insufficient for compression of shared in- formation. Moreover, also need avoid the damage of over-compression for information validity, i.e., the learned joint representation z lacks sufficient information for pre- diction. Based on this, we propose to maximize the mutual information between the original data and the joint repre- sentation:
These methods directly use the generated hard pseudo-labels to train the model, treating the generated pseudo-labels ground-truth. Since pseudo-labels are not entirely accu- rate, the use of hard pseudo-labels may cause the model to deviate, further amplifying errors and leading to a vicious cycle. To address this problem, we propose a dual-branch soft pseudo-label cross-imputation strategy to reduce the hindering effect of multi-label missing on the multi-label classification.
max I T(x):
(5)
Obviously, Eq. (5) requires that the cross-view joint repre- sentation maintains mutual information maximization with the original data of all views. It ensures that the cross-view joint representation z further learns the shared information across all available views.
3. Method
In this section, we introduce our method in three parts: in- complete mueti-view contrastive learning; compact shared information representation learning based on mutual infor- mation maximum; dual-branch soft pseudo-label generation and multi-label classification.
For mutual information T(XY; in Eq. (5), obtain the variational lower bound as follows:
1(x
/ p(XV, z) log
p(x (z)
dx''dz dz
v(XV)
(6)
3.1. Incomplete Multi-view Contrastive Learning
2 Exy=pxrnn/ p(z|X") v log q(X X (2)ddz
According to the analysis in Section 2, we first establish an optimization objective to maximize mutual information between views, thereby promoting multi-view consistent semantic representation learning. Given any two views, the optimization objective can be expressed as:
where (x) is an additional distribution introduced to approximate p(X) And then, based on the multi- view conditional independence assumption, conditional dis-
max ΣΣ
(0)
tribution ((x))z) can be decomposed into gX' 1)) =
II ((x()) (z). Combined with Eq. (4), we can rewrite
VEV Eip
4

---

COME: Incomplete Multi- View Missing Multi-Label Classification
Raw data: (x(1)
R
Cross-viaw information
generation
View
Views View 3
Reconstruction data
Raw data
Reconslruction data
View
View 3
Threshold fittering
View
View
View
p
CAS
Cuei
Luxa
Embedding:
R
View
Fusion
Thrushold
Incomplate multi contrastive learning
View specific encoder
Sumple
View?
Vied
View
LeIR
g+
Label predicator
VView specific decoder
Sumete
A
Known label
Known view set
Sample
Stop-gradient
Known label set
10 Incomplete view
O
A Complete view
Push
Pull
Soft pseudo-label
Figure 3. Our main framework of COME, consists of incomplete multi-view contrastive learning, multi-view mutual information enhancement, and dynamic soft pseudo-label generation.
the variational lower bound in Eq. (5) as follows:
ing:
LMSR = XI + A2CRE2 + BLTTR
(9)
Exv-pxxvll p(z|x)) log a( [())
where is the penalty coefficients for contrastive loss.
By Exxe) Bxtempixeoss p(z)x((1) log (x((') (z)dz]
VEV
3.3. Dual-Branch Soft Pseudo-label Generation and Multi-Label Classification
E Σ VEV uEV.Nfo
B spxcorl / p(z)x(()) log (") [z)dz]
Previous works directly use the pscudo-labels generated by the model as supervision for training in the next epoch. However, argue that using the current network's high- confidence predictions ground truth provides limited ben- efit to the network. It may also lead to overfitting on simple samples, as the model's 's performance bottleneck does not lie in the classification of easily distinguishable samples. More- over, this method of using pseudo-labels amplifies the mis- leading effect of incorrect pseudo-labels and errors will con- tinue accumulate during training. To address this, we pro- pose dual-branch soft pseudo-label cross cross- imputation strat- egy to efficiently generate reliable pseudo-labels, thereby further enhancing classification performance.
The full derivation of Eq. (6) and Eq. (7) is given in Ap- pendix A. Here, we use (xx(") |z) as an approximation of p(x(*) z), which is implemented by a stochastic decoder with constant standard deviation as Specifically, we set q(x(=) 1)) ((x(")//(()) (z), I), where p!()) the mean mapping function. Combined with the characteristics of Gaussian distribution, we have the equivalent of the op- timization objective: max LE (2) [log ((x(*) z)] <
max- - [ [I|x(*) ) (())) (z) Thus, it is easy to get the following reconstruction loss for maximizing Eq. (7):
CRE = A1 Σ [xx (z())))
Specifically, we first use a dual-branch model to gen- erate and utilize pseudo-labels, employing two indepen- dent but structurally identical models. and i.e., A: (x(*) pm - - A B: {x J + P.. where d and p are the predicted logic scores by model A and B. By doing this, we build two independent prediction models as two branches. In light of the limitation of the pseudo-label self- self-generation strategy Section 2.4, we pro- pose to leverage the external pseudo-label to improve the optimization of the model. Briefly, our core motivation
NEV
A
(x(u)_ -000)~((((l0 (z()))
(8)
VEV WE V.ueo
>iERE 22LRE
where Dj and Xe are trade-off coefficients. Then, we com- bine the reconstruction loss and the contrastive loss to achieve the goal of minimal sufficient representation learn-
5

---

7: Calculate the incomplete multi-view contrastive loss, inter-view reconstruction loss and intra-view recon- struction loss by Eq. (9), and then obtain LMSR- COME: Incomplete Multi- View Missing Multi-Label Classification
to have two branches generate pseudo-labels for each other for the round of iteration. To he specific, pseudo-label generation process from model A is as follows:
Algorithm 1 Training process of COME
1: Input: Incomplete multi- data {x((*) }== known
view set V. incomplete multi-label y E {0,1]- known label set g.
None, Pe
if E 9 and p < T
2: Output: Trained parameters of model A.
GA =
if c & 9 and p 2 (10)
3: Initialization: Initialize the parameters of the model
Ye:
if
A and set hyper-parameten (21, 22, 8, and training epochs E)
where ý denotes the pseudo-label from model A. In addi- tion, we need to incorporate the generated soft pseudo-labels into the known label set and adjust the known label set g corresponding to A.A and B to get رA and GB, respec- tively. To enhance the reliability soft pseudo-labels, we propose to build dynamic threshold adjustment strategy. During the initial phases of training, more stringent thresh- old is employed to ensure the high quality of pseudo-labels, thereby minimizing the risk of introducing noise. As the training progresses, the threshold is gradually relaxed to ex- tend the range of generated pseudo-labels, striking a balance between accuracy and error:
4: for e: = 1,..., F do
5: Compute samples latent representation {z(") Je== on each available view by encoders.
6:
Reconstruct available views by decoders.
8: Fusion multi-view representation of samples by Eq. (4) and get multi-view joint representation z. Obtain y from model B by Eq. (10), and obtain the corresponding new known label set ς
10: Calculate the classification loss LCg of model A by Eq. (12), and compute total loss of model A: C LCE + L.M
I = max(n, Th - e x 0)
(11)
where 77 denotes the lower bound of the threshold range and Th represents the initial largest threshold, 0 the threshold decay rate and denotes the e-th epoch. With the two generated soft pseudo-labels ý and yB, we simply swap them as the supervised signal for the next training epoch of the other branch. For model A, the cross-entropy loss is as following:
11: Update the parameters of model A with C.
12:
end for
multi-label datasets, i.e., Corel5k (Duygulu et al., 2002), Pasca107 (Everingham et al., 2010), ESPGame (Von Ahn & Dabbish, 2004), IAPRTC12 (Grubinger et al., 2006), and Mirflickr (Huiskes & Lew, 2008). Each dataset comprises six distinct feature types: GIST, HSV, DenseHue, Dens- eSIFT, RGB, and LAB. The detailed information about the five datasets and the data masking schemes that used to em- ulate real-word scenarios can be found in the Appendix B.
1
LCE
[11 jEğn
Σl(
})) log(1 - pT*)
5B)
|
+y,Pog
log 1 (12)
Our two-branch networks generate pseudo-labels for each other, and for each one, this strategy helps it to obtain ex- tra supervisory information from outside. The training ap- proach and parameter update strategy for both branches are same but independent. Since the structure of the two branches is the same, we simply take the prediction of model A as the final result. Finally, we simply sum the objective functions of each component to obtain the following overall optimization objective:
Comparison methods: In our experiments, we select eight popular methods for comparison with our COME. The se- lected methods include: CDMM (Zhao et al., 2021), DM21. (Ma & Chen, 2021), I). LVSL (Zhao et al., 2022), iMVWL (Tan et al., 2018), NAIM3L (Li & Chen, 2021), DICNet (Liu et al., 2023), UPDGD-Net (Xie et al., 2024b), SIP (Liu et al., 2024). However, not all methods incomplete views and labels, necessitating modifications to certain approaches. We applied specific modifications to different models, and further technical specifics can be found in Appendix C.
capable of handling
L ARER + >2CREZ+ + 3LCTR + Lc
(13)
where '₁, A2 and 3 are hyper-parameters. For ease of un- derstanding, we provide the training process of model A in Algorithm 1.
Evaluation metrics: To ensure consistency with previous work, we employed six popular performance metrics: Av- erage Precision (AP), Hamming Loss (HL), Ranking Loss (RL), Area Under the ROC Curve (AUC), OneFirror (OE), and Coverage (Cov). To better present the results, we use 1-RL, 1-HL, 1-OE, 1-Cov as the final result for demonstra- tion ensuring that higher values indicate better performance across all six metries.
4. Experiments
4.1. Experimental Settings
Datasets: In line with previous works (Tan et al., 2018; Liu et al., 2023), we conduct experiments on five multi-view
6

---

COME: Incomplete Multi- Missing Multi-Label Classification
Table 1. Experimenntal results of nine methods five datasets with 50% missing-viert rate and 50% missing-label rate (the hottom right digit is the standard deviation).
Data Metrie
CDMM
DM2L
LVSL
iMVWL
NAIM3L
DICNet
UPDGD-Net
SIP
COME
AP
0.3540.004
0.2622.o0s
0.342o.o0z
0.2833.cm o0s
0.309
0,381o 004
0.4130.000
0.418o.om
0,432o 000
I-HT.
0.9870.000
0.987a.000
0.987o.000
0.9780.000
0.9870.000
0.988p.oe 000
0.9870.000
0.988a.000
0.988o.00
Core15K
I-RL
0.8840.00s
0.843k.002
0,881a.00 o0s
0.865g.000
0.8780.002
0.882p oos
0.903
0.91 1901l..0s
0.9176.00
AUC
0 18888..00
0.8455.on2
0.884o.oos
0.868a.00s
0.88 0.881p.oce
0.884o.oo 0o4
0.905o.co 000
0.913o.cr 300
0.919b.0c
1-OE
0.4100.007
0. 2955..04
0.391o.009
0.31 10.01
0.350p.
0. 0,468o.um ou7
0.480o.ooz
0. 0.4890.o16
0.5030.00
I-Cov
0 0.723
0.647o.ous
0.7 18718000
0.702a.008
0.725p.00s
0.7270.011
0.7770.00s
0.787o.oue
0.804g.ge
AP
0.508
0.471o.008
0.5044.00s
0.437a.018
0,488p.ock o00s
0.505p.o12
0.552
0.555o.010
0.590g.oo oos
I-HLL
0.9310.001
0.928o.001
0,930a.000
0.882a.00
0.928p.no
0.929%..0
0.933o.007
0,931o.001
0,935a.cm
Pascal07
I-RL
0.8 0.8120.004
0.761o.o05
0.8066.0m8
0.7366.015
0.783g.u0
0.783o pos
0. 0.8320.com .007
0,830o.004
0.855o 004
AUC
0 0.838
0.7799.004
0.832a.002
0.767a.013
0.81
p.001
0.809 ooG
0.8536.00m
0.850..or
0.873 .003
I-OE
0.4199.008
014
0.419u.008
0.362..023
0.421o.
0.427p.015
0.461o.007
0.4644..18
0.500 010
I-Cov
0. 0.7590.008
0.692o.on4
0.751a.00s
0.677a.015
0.7276.002
0.731o.000
0. 0.7855..00
0.783o.006
0.805a.od
AP
0.2890.003
0.2122.002
0.285o.ou3
0.244g.00s
0.246g.002
0.297p. 12970.002
0.312o.004
0.3110.004
0.3199p.0o
ESPGaaee
I-HL.
0.9836.000
0.9820.000
0.9830.000
0.972a.000
0.9830.000
0.983 0.000
0.9836.000
0.0830.000
0.983..0v
I-RL
0.832o.001
0.78 10.001
0.829..001
0.808
0,818a.002
0.832 001
0.8477.002
0.849..002
0.857a.002
AUC
0.8366.001
0.7850.001
0.833o.002
0.813a.002
0.824a.on2
0.8366.003
0.852o.002
0.853o.002
0.861a.002
1-OE
0.3960.005
0. 0009
0.3896.004
0.343a.013
0.339a.003
0.439u. 0077
0 0.461o.010
0.4550.007
0.460o.az 010
I-Cov
0.5740.004
0.4880.003
0.5670.006
0.548g.004
0.57
0.593
0.628
0.6288.o06
0.647a.004
AP
0.305..004
0.2346.003
0.3044.004
0.261g.001 0.2377.00s
0.323 001
0.339
0.331o.000
0.35
IAPRICI2
1-HL
0..800
0,9806.000
0.981a.000
0.969a.000
0,980a.000
0,981 0.000
0.980p.oo0
0.980p.o00
0,981a.000
1-RL
0. 0.8622.002
0.023
0.861o.002
0.833o.0u2
0,848p.001
0.873p.00 0.03
0. 1886o.002
0.885
0,8055.000
AUC
0.8640.c02
0.825o.001
0.8630.001
0.835a.001
0.850g.001
0.874p.0o0
1.8880.004
0.8866.002
0.896g.002
1-OE
0.432o.00s
0.3400.000
0.429..009
0.3522.008
0.390u..05
0.468 0022
0.4630.002
0.4633.009
0,486 oos
I-Cov
0.5970.004
0.5296.004
0.5976.004
0.5640.o0s
0.592a.064
0.649p.001
0.688o.007
0.6750.007
0.700p.0d
AP
0.570p.co2
0.5 0.5140.006
0.553o.o0z
0.490s.012
0,55 10.002
0.589p.003
0.61lo.ooz
0.6140.004
0,6333 .0o4
I-HL
0.8866.001
0.8780.001
0.885a.001
0.8399.002
0.882a.001
0. 18888 0022
0.89 1o.001
0.89 10.001
0.895 0011
Mirlickk
I-RL
0.8566.001
10.33
0.856..001
0.803
0.8444.001
0.863 004
0.875
0.888, oo2
AUC
0. 0.8466.001
0.828o.00s
0.844..001
0.787
0,837a.001
0. 0.849p.004 004
0. 1862o.001
0.860p.ons
0.874g.002
1-OE
0.6310.004
0.5 0.510b.00s
0.6070.004
0.51 1o.0z2
0.585g.om3
0.637p.007 0.077
0. 166622.006
0.662o.ous
0.683g.0os
1-Cov
0.640o.co1
0.6040.005
0.6366.001
0.572a.013
0.63 1a,002
0.652p.007 n07
0.681 1.68
0.678o.o0a
0.694g.003
4.2. Experimental Results and Analysis
In this section, compare method with other eight pop- ular algorithms on the five datasets mentioned above and the experimental results under 50% missing- view rate and 50% missing-label rate of the six evaluation metrics are shown in Table 1, and we can have the following observations:
Valuee of four movics
0.5
0.5
0.4-
wissing-iin rate
0.4
enssteg oakee rate
0%
0%
30%
30%
50%
50%
Compare to other eight popular methods on the five datasets, our COME achieves the best performance on all metrics which fully verifies the effectiveness of our method on iM3C task.
(a) various missing view rates
(b) various missing label rates
Figure 4. The four metrics value on Pascal07 dataset with (a) dif- ferent missing-vies view rates and 50% missing-label rate and (b) 50% missing-view rate and different missing-label rates.
Compared to models that only consider single missing data, models that address the dual missing problem exhibit better performance and robustness. These find- ings offer valuable insights for guiding the design of models for iM3C tasks the future.
missing rate, view missing has a more severe impact on performance improvement compared to label missing.
To further validate the adaptability of our model, we con- ducted experiments using datasets without any missing views and labels. The results on the five dataset are illus- trated in the form of a radar chart, as shown in Fig. 5 (refer to Appendix F for more results on other databases). It can be observed that our COME achieves superior performance across six metrics compared to other methods, including
To further investigate the impact of different missing view and missing label ratios on classification performance, we conducted tests on the Pascal07 dataset. The results are shown in Fig. 4. It can be observed that, under the same
7

---

COME: Incomplete Multi- Missing Multi-Label Classification
1:4
CERM
10.354 0.942
(..07.0.000)
[0/90, 0.00)
[MC2
2.606,2911 23101
6.00e,
10040 0.000
DNGL
COPE
TAUG
T-LLIC
DONG
20,00, 0.0%g
2.700)
1:cE (8.35) 6.030)
S-Cee
140
10W
11/30 2811)
patt 2.74
(a) Corel5k
(b) Pasca107
(c) Mirflickr
Figure 5. The experimental results nine methods on the complete Corel5k, Pascal07, and Mirflickr dataset. this visualization, the worst results positioned at the center, while the hest results correspond to the vertices, based six evaluation metrics.
those designed for ideal complete cases. This demonstrates the excellent generalization capability of our model.
Table 2. The ablation experimental results on the Corel5K and Pasca107 datasets, and all datasets are with 50% missing views, 50% missing labels and 70% training samples.
4.3. Shared Information Compression v.s. Information Reconstruction
CorelSk
Pasca107
As we know, contrastive learning is typical information compression strategy, while our proposed mutual informa- tion enhancement strategy based on MoE emphasizes the maintenance of effective shared information. That is to say, the learned shared information remains relevant to the origi- data to prevent information collapse. In this section, we study the equilibrium relationship between CREI CRE2 and LсTa a by changing the value of parameter B. As shown in Fig. 6, when the value of B3 is 0.1 and 1 for Corel5k and Pasca107 datasets, respectively, information compression and effective information reconstruction reach a balanced state, and the model achieves the optimal performance. If we continue to increase B, we can find that AP value de- creases sharply, which indicates that excessive information compression caused by contrastive learning makes it dif- ficult for the model to learn meaningful information for downstream prediction.
Backbone
CREI
Los
AP
AUC
AP
AUC
0.390
0.883
0.535
0.846
0.399
0.908
0.581
0.868
0.402
0.910
0.562
0,862
v
0.390
0.903
0.538
0.847
0.427
0.918
0.586
0.872
0.406
0.912
0.584
0,871
0.404
0.910
0.565
0.863
0.432
0.919
0.873
4.4. Ablation Study
The ablation experiments are conducted on Corel5k and Pas- cal07 datasets, which the missing- view rate and missing- label rate are both 50%. Our objective function consists of two parts: the minimal sufficient shared representation learning loss LMR and classification loss Lo Next, we will decompose the loss and conduct ablation experiments from three perspectives: examining the effects of intra-view mutual information enhancement, multi-viee mutual infor- mation enhancement, and pseudo-labels The corresponding loss of these three parts are LREI, CREZ. and Log. It is worth noting that when pseudo-labels are not used, the su- pervisory information for the model comes from the original labels that are not missing. The results presented in Table 2, and it can be observed that each component of our COME is crucial and contributes positively. is worth noting that the multi-view mutual information enhancement module plays a crucial role in improving performance.
Eoe
10
10
(a) Corel5k
(b) Pascal07
5. Conclusion
Figure 6. Information balance study: AP and losses v.s. 3 on Corel5k and Pascal07 datasets with 50% view and label missing rates.
In this paper, we assume that the consistent semantic infor- mation shared among views is sufficient for downstream tasks. Based on this, we propose a compact semantic learn ing framework, named COME, for iM3C task. Our moti-
8

---

COME: Incomplete Multi- View Missing Multi-Label Classification
vation to this issue consists of two parts: compact shared representation learning and soft pseudo-label generation. On the one hand, we alleviate the limitations of contrastive learning in handling incomplete data by maximizing mu- tual information between original data and cross-view joint representation, striving to learn the compact shared repre- sentation. On the other hand, we propose dual-branch soft pseudo-label cross-imputation strategy to mitigate the impact of missing supervisory information. Finally, con- ducted extensive experiments to validate the effectiveness of our method and its strong generalization capability.
ing lexicon for fixed image vocabulary. In Computer Vision-ECCCV 2002: 7th European Conference Com- puter Vision Copenhagen, Denmark, May 28-31, 2002 Proceedings, Part IV 7, pp. 97-112. Springer, 2002.
Everingham, M., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. The pascal visual object classes (voc) challenge. International journal of computer vision, 88: 303-338, 2010.
Fallah, H., Bruno, E., Bellot, P., and Murisasco, E. Exploit- ing label dependencies for multi-label document classi- fication using transformers. In Proceedings of the ACM Symposium on Document Engineering 2023, pp. 1-4, 2023.
Acknowledgments
This work was supported in part by Shenzhen Sci- ence and Technology Program under Grant No. in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2024A1515030215 and in part by National Natural Science Foundation of China under Grant 62372136.
Fang, C., Sandino, C., Mahasseni, B., Minxha, J., Pouransari, H., Azemi, E., Moin, A., and Zippi, E. Pro- moting rross-modal representations to improve multi- modal foundation models for physiological signals. arXiv preprint arXiv:2410. 16424, 2024.
Fang, X., Liu, D., Zhou, P., and Hu, Y. Multi-modal cross- domain alignment network for video moment retrieval. IEEE Transactions on Multimedia, 25:7517-7532, 2022.
Impact Statement
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Fang, X., Liu, D., Zhou, P., Xu, Z., and Li, R. Hierarchical local-global transformer for temporal sentence grounding. IEEE Transactions Multimedia, 26:3263-3277, 2023,
References
Fang. X., Easwaran, A., Genest, B., and Suganthan, P. N.
Your data is not perfect: Towards cross-domain out-of- distribution detection in class-imbalanced data. Expert Systems with Applications, 267: 2025.
Ameer, I., Sidorov, G., Gomez-Adorno, H., and Nawab, R. M. A. Multi-label emotion classification on code-mixed
text: Data and methods. IEEE Access, 10:8779-8789, 2022.
Federici, M., Dutta, A., Forré, P., Kushman, N., and Akata, Z. Learning robust representations via multi-view in- formation bottleneck. arXiv preprint arXiv:2002.07017, 2020.
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N.,
Oliver, A., and Raffel, C. A. Mixmatch: A holistic ap- proach to semi-supervised learning. Advances in neural information processing systems, 32, 2019.
Gidaris, S., Singh, P., and Komodakis, N. Unsupervised rep- resentation learning by predicting image rotations. arXiv preprint arXiv:1803.07722 2018.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International conference on machine learning, pp. 1597- PMLR, 2020.
Grubinger, M., Clough, P., Müller, H., and Deselaers, T. The iapr te- 1c-12 benchmark: A new evaluation resource for visual information systems. In International workshop ontolmage, volume 2, 2006.
Chen, Z.M.., Wei, X.-S., Wang, P., and Guo, Y. Multi-label image recognition with graph convolutional networks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 5177-5186, 2019.
Huiskes, M. J. and Lew, M. S. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference on Multimedia information retrieval, pp. 39- 43, 2008
Devlin, J. Bert: Pre-training of deep bidirectional trans-
formers for language understanding. arXiv preprint 1011::1810.04800 2018
Hwang, H., Kim, G.-H., Hong, S., and Kim, K.-E. Multi- view representation learning via total correlation objec- tive. Advances in Neural Information Processing Systems, 1112194-12200 2021.
Duygulu, P., Barnard, K., de Freitas, J. F., and Forsyth, D. A. Object recognition as machine translation: Learn-

---

COME: Incomplete Multi- View Missing Multi-Label Classification
Khosla, P., Teterwak, P., Wang. C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. Supervised contrastive learning. Advances in neural information processing systems, 33: 11111166618775 2020.
Shi, Y., Paige, B., Torr, P., et al. Variational mixture-of- experts autoencoders for multi-modal deep generative models. Advances in neural information processing sys- tems. 32, 2019.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., and Li, C.-L. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural informa- tion processing systems, 33:596-600 2020.
Lee, D.-H. et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural net- works. In Workshop on challenges in representation learning, ICML., volume 3, pp. 896. Atlanta, 2013.
Soleymani, M., Garcia, D., Jou, B., Schuller, B., Chang, S.-F., and Pantic, M. A survey of multimodal sentiment analysis. Image and Vision Computing, 65:3-14, 2017.
Li, X. and Chen, S. A concise yet effective model for non- aligned incomplete multi-view and missing multi-label learning. IEEE Transactions Pattern Analysis and Machine Intelligence, (4(10))5918-50) 5932, 2021.
Tan, Q. Yu, G., Domeniconi, C., Wang, J., and Zhang, Z. Incomplete multi-view weak-label learning. In Ijeai, pp. 2703-2709, 2018.
Thomas, M. and Joy, A. T. Elements of information theory. Wiley-Intersciene 2006.
Lin, Y., Dou, X., Luo, X., Wu, Z., Liu, C., Luo, T., Wen, J., Ling, B. W.-k., Xu, Y., and Wang, W. Multi-view diabetic retinopathy grading via cross-view spatial alignment and adaptive vessel reinforcing. Pattern Recognition, 164: 111487, 2025.
Tian, Y., Krishnan, D., and Isola, P. Contrastive multiview coding. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceed- ings, Part XI 16, pp. 776-794. Springer, 2020.
Liu, C., Wen, J., Luo, X., Huang. C., Wu, Z., and Xu, Y. Dic- net: Deep instance-level contrastive network for double incomplete multi-view multi-label classification. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 37, pp. 8807-8815 15,, 2023.
Tsai, Y.H.. H., Wu, Y., Salakhutdinov, R., and Morency, L.- P. Self-supervised learning from a multi-view perspective. arXiv preprint arXiv:2006.055: 05576, 2020.
Von Ahn, L. and Dabbish, L. Labeling images with a com- puter game. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 319-326, 2004.
Liu, C., Xu, G., Wen, J., Liu, Y., Huang, C., and Xu, Y. Partial multi-view multi-label classification via semantic invariance learning and prototype modeling. In Forty-first International Conference on Machine Learning, 2024.
Wang, H., Guo, X., Deng, Z.H.., and Lu, Y. Rethinking min- imal sufficient representation in contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16041-16050, 2022.
Luo, X., Liu, C., Wong, W., Wen, J., Jin, X., and Xu, Y. Mveinn: multi-view diabetic retinopathy detection using a deep cross-interaction neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 37, pp. 8993-9001, 2023.
Wen, J., Zhang, Z., Zhang, Z., Wu, Z., Fei, L., Xu, Y., and Zhang, B. Dimc-net: Deep incomplete multi-view clustering network. In Proceedingg of the 28th ACM international conference on multimedia, pp. 3753-3761, 2020.
Ma. Z. and Chen, Expand globally, shrink locally: Dis- criminant multi-label learning with missing labels. Pat- tern Recognition, 1 111:107675, 2021.
Wen, L., Nie, M., Chen, P., Zhao, Y.n.,, Shen, J., Wang, C., Xiong, Y., Yin, K., and Sun, L. Wearable multimode sen- sor with a seamless integrated structure for recognition of different joint motion states with the assistance of a deep learning algorithm. Microsystems & nanoengineering. 8 (1):24, 2022.
Pan, F. and Kang, Z. Multi-view contrastive graph cluster- ing, Advances in neural information processing systems, 14:2148-2150 2021.
Qiu, S., Miller, M. I., Joshi, P. S., Lee, J. C., Xue, C., Ni, Y., Wang, Y., De Anda-Duran, I., Hwang, P. H., Cramer, J. A., et al. Multimodal deep learning for alzheimer's 's disease dementia assessment. Nature communications, 13(1) (31444404, 2022.
Wu, X., Chen, Q.-G,, Hu, Y., Wang, D., Chang, X., Wang, X., and Zhang, M.-L. Multi-view multi-label learning with view-specific information extraction. In IJCAI, pp. 2019.
10

---

COME: Incomplete Multi- Missing Multi-Label Classification
Xiao, J.-H., Xie, M.-K., Fan, H.-B., Niu, G., Sugiyama, M., and Huang, S.-J. Dual-decoupling learning and metric- adaptive thresholding for semi-supervised multi-label learning. arXiv preprint arXiv:2407.18624 2024.
Zhang, Y., Wu, J., Cai, Z.., and Philip, S. Y. Multi-view multi- label learning with sparse feature selection for image annotation. IEEE Transactions Multimedia, 22(11): 1844-2857, 2020.
Xie, M.-K., Xiao, J., Liu, H.-z., Niu, G., Sugiyama, M., and Huang, S.-J. leias-distribttin-awaana pseudo-labeling for semi supervised multi-label learning. Advances in Neural Information Processing Systems, 36, 2024a.
Zhao, D., Gao, Q., Lu, Y., Sun, D., and Cheng, Consis- tency and diversity neural network multi-view multi-label learning. Knowledge-Baad -aseed Systems, 218: 118:100 106841, 2021.
Zhao, D., Gao, Q., Lu, Y., and Sun, D. Non-aligned multi- view multi-label classification via learning view-specific labels. IEEE Transactions Multimedia, 25:7235-7247, 2022.
Xie, W. Lu, X., Liu, Y., Long, J., Zhang. B., Zhao, S., and Wen, Uncertainty-aware pseudo-laheling and dual graph driven network for incomplete multi-view multi- label classification. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 6656-6665, 2024b.
Zhou, Z.-H., Zhang, M.-L.,, Huang, S.-J., and Li, Y.-F Multi- instance multi-label learning. Artificial Intelligence, 176 (1):2291-2320, 2012.
Xu, Y., Shang, L., Ye, J., Qian, Q. Li, Y.-F., Sun, B., Li, H., and Jin, R. Dash: Semi-supervised learning with dynamic thresholding. In International conference on machine learning, pp. 11525-11525 11536. PMLR, 2021.
Yang, W., Zhang, R., Chen, J., Wang, L., and Kim, J. Prototype-guided pseudo labeling for semi-supervises text classification. In Proceedings of the 6lst Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 16369-16382, 2023a.
Yang, 7., Xu, Q., Bao, S., Cao, X., and Huang, Q. Learning with multiclass auc: Theory and algorithms. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 44 (11):7747-7763, 2021.
Yang. Z., Xu, Q. Bao. S., He, Y., Cao, X., and Huang, Q. Optimizing two-way partial auc with an end-to-end framework. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8); 10228 10246, 2023b. doi: 10. 2.1109/TPAMM..222.11555
Zeng. P., Yang, M., Lu, Y., Zhang, C., Hu, P., and Peng, X. Semantic invariant multi-view clustering with fully incomplete information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Zhang, C.,, Liu, Y., and Fu, H. Ae2-nets: Autoencoder in autoencoder networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2577-2585, 2019.
Zhang, D., Ju, X., Zhang, W., Li, J., Li, S., Zhu, Q. and Zhou, G. Multi-modal multi-label emotion recognition with heterogeneous hierarchical message passing. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 35, pp. 14338 14346, 2021.
Zhang, H., Wang, Y., Yin, G., Liu, K., Liu, Y., and Yu, T. Learning language-guie guided adaptive hyper-modality representation for multimodal sentiment analysis. arXiv preprint urXiv:2310. 100:2300.058800 0.055044 2023.

---

COME: Incomplete Multi- Missing Multi-Label Classification
A. Complete Derivation of Shared Representation Learning
Based on the analysis in Section 3.2, we propose to maximize the mutual information between the original data and the joint representation:
max T(x"; :z)
(14)
For mutual information I(X"; :z) in Eq. (14), we can obtain the variational lower bound as follows:
T(X²:0) :2)
1 P(X) .2) log
p(X (2) dx
dz
p(X))
2 M(X)) / p(z)x") log p( X (2)dx de - pix)) / plzXx log logg(X (003((((12)))))) (z) dz +
(15)
/ m(xx) / plz(X") log
p(X (z)
log
dx dz
q(X V
(z)
2 Exv~pox) p(z|X)) log ((X V [z)
Similar previous work(Shi al., 2019), we propose to factorise the joint variational posterior as combination of unimodal posteriors, using Mixture Of Experts (MOE) to model p(z) ()), therefore have:
p(2[
Σ DEV 1 P(2
(x
(16)
Combined with Eq. (16), rewrite the variational lower bound in Eq. (15) follows:
Exv Exv Exvini / p(z|x>) ) log ((x V [z)dz]
=Ex = / p(z)x*)
log II ((x(()/2)dd)
-Bxm~pixn1
/
1u p( (x (0))))) log II ((x("))2)2)
(17)
E Extrrpp(tt) / p(z)x(()) log ((x(())))())) [z)dz]
M
Σ
E Exterteccon
/ p(z)x(")) log ((x(() (2)dz]
For better clarity, we will use two views as an example for the subsequent derivation. We have the joint posterior probability p(z)(x!) ((())))))).) And the variational lower bound can be rewrite as follows:
/ x(z)x log q(X lz)ddz)
Ex Ex
Ex Ex (p(z)x!))
((g +2)))))
Exx= (D(z)(())
+ (p(z|x ( loglg(x (x)) |z)]
Ex ExX~mxx)!! p(z)x(1) log x(x ((xx 12)d2z / p(z|x(1) log ((x ((((2)) (2)dz
2
/ (2))x(2) (2o(2)))oggt (o / p(z)x(2) log (x z)dz)
(18)
12

---

COME: Incomplete Multi- Missing Multi-Label Classification
Table 3. Detailed information about five multi-view multi-lahel datasets our expriments.
Dataset
Sample
# Label
View
# LLabel/#Sample
Corel5k
4999
260
6
3.40
IAPRTC12
19627
291
6
5.72
ESPGame
20770
6
4.69
Pascal07
9963
20
6
1.47
Mirflickr
25000
38
6
4.72
Table The concise summary of comparative methods, highlighting their capabilities: The attributes 'Multi-view" and 'Incomplete- view" indicate method's ability handle multi-view and incomplete multi-view scenarios, respectively, while the attribute "Missing-label" reflects its capability to address incomplete multi-label classification tasks.
Method
Sources
Multi view
Incomplete-viee
Missing-labe -abel
CDMM
KBS'20
DM2L
PR'21
LVSL
TMM 22
X
iMVWL
ICCAI18
NAIM3L
TPAMI'21
DICNet
AAAI'23
UPDGD-Net
ACM MM'24
SIP
ICML'24
B. Statistics for Five Datasets and Data Masking Schemes
In this section, we provide detailed information about the five datasets used in the experiments, summarized in Table 3. To emulate real-world scenarios involving missing views and partial labels, we generate incomplete multi-view partial multi-label data through the following steps: (1) Missing views: For each view, 50% of the samples are randomly masked, while ensuring that each sample retains at least one available view. (2) Missing labels: For each category. 50% of the positive and negative labels are randomly removed. (3) Dataset Splitting: Subsequently, 70% of the resulting samples are randomly selected as the training set.
C. Statistics for Eight Competitors
In this section, we present detailed information about the eight comparison methods, as summarized in Table 4. Specifically, DM2L can only deal with single-view partial multi-label case, so we record results of each view and select the best results. CDMM and LVSL cannot deal with missing view, so we handle missing instances by imputing them with the mean values of the corresponding view's 's available instances. And the rest five methods have the capability handle both incomplete views and partial labels.
D. Sensitivity Studies of Hyper-parameters
We discussed enhancing the performance of learned representations through two approaches: increasing intra-modal mutual information and inter-modal mutual information. Additionally, we use two trade-off coefficients A₁ and A2 to maintain a balance. We explore the sensitivity of these two hyperparameters on different datasets with 50% missing view, 50% missing label and 70% training samples. As shown Fig. 7, based on experiments, can be observed that the model's performance not significantly affected by changes in the values of X and X2, highlighting the model's inherent robustness.
E. Label Count Distribution across Samples in Other Datasets.
In this section, we present the label count distribution information for other datasets. This further demonstrates that the number of labels corresponding to each sample is not fixed, making Top-1 or Top-k pseudo- label generation strategies difficult apply.
13

---

COME: Incomplete Multi-View View Missing Multi-Label Classification
03
03
22
02
0.1
91
100
10
700
100
125
300
150
Core15k
(b) Pasca107
(c) IAPRTCI2
Figure 7. Hyper-parameters analysis regarding A and A2 on the Corel5k and Pascal07 databases; All datasets are setting to 50% views and labels missing
0.15
0.12
Proportion
0.3
0.4
0.12
0.15
Proportion
0.08
0.10
0.06
0.2
0.08
0.05
0.04
0.1
0.03
2
9 10 Number of categories per sample
3
1 7 8 101112
1 4 5 6
8 9 1011
Number of categories per sample
Number of categories sample
Number of categories per sample
(a) Corel5k
(b) ESPGame
(c) IAPRTC12
(d) Mirflickr
Figure 8. Label count distribution across samples in other four datasets.
F. Extra Experimental Results on Five Full Datasets
In this section, we present the value of six metries of nine methods four datasets without any missing views or labels in the form of radar charts, as shown in Fig.
1-R1 10.817, 0.882]
1-H
[0.972 0.904)
Jo 0.921)
D92L
DSL
MAMOL
DiClet
DIClet
3P
LPOGD
CONF
J0,82,
|0.2011 0.391]
[0,84], 0:529)
[0,250, 0.424)
1-DE Jo,400 0.540]
Cox (0 586, 0 5050
1-EE (0.348, 0 5888)
[0.581 0.783)
(a) ESPGame
IAPRTC12
Figure 9. The experimental results the rest two dataset without any missing views and labels. The worst results are indicated at the center the radar chart, while hest results are represented by vertices, based on six evaluation metrics.
14


Reviews:
[{"review_id": "9qJPTZxwQK", "reviewer": "Reviewer_abe3", "summary": "This paper proposes a multi-view multi-label learning approach by integrating compact semantic information learning and pseudo-labeling imputation to address the degenerative multi-view contrastive learning and missing label issues. The authors elaborate the failure of multi-view contrastive learning in view absence and introduce the compact semantic information extraction framework. Moreover, the soft label filling approach is used to improve classification performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable suggestions! Below, we will address each of your questions.\n> Q1: Providing comprehensive experimental details (e.g., dataset-specific hyperparameters) or releasing the source code would significantly enhance the transparency of this work.\n\nThank you for your suggestion. We will release the code and pre-trained checkpoints upon acceptance. For reproducibility we build on the open source implementation of COME.\n> Q2: Some statements are not clear enough: on Equation (16) and line 625, “Since we adopt the MoE fusion strategy to model $p(z\\vert X^V)$, we have:” which is not very clear.\n\nWe sincerely apologize for the confusing. Similar to previous work [1], we propose to factorise the joint variational posterior as a combination of unimodal posteriors, using a Mixture Of Experts (MOE), therefore we have Eq. (16).\n> Q3: In the paper, the pseudo label generation models are able to effectively mitigate label missingness. However, beyond the ablation studies, I recommend further investigation into the efficacy of this claim.\n\nThank you for your suggestion. We conducted an in-depth investigation of the dual-branch soft pseudo-labeling strategy under varying label-missing rates on the Pascal07 dataset, with the corresponding four metrics ​presented in Figure 3 of the PDF document ​provided via the anonymous link. The experimental results demonstrate that the dual-branch soft pseudo-labeling strategy exhibits superior performance under elevated label-missing conditions, particularly when the missing rate is 70%.\n> Q4: (Line 88) The period before “(Federici et al., 2020; Tasi et al., 2020)” should be removed. Double check for this manuscript is crucial.\n\nWe appreciate the reviewer’s careful attention to detail. The period before the citations in line 88 has been removed in the revised manuscript to adhere to proper punctuation conventions. Thank you for highlighting this oversight.\n\n> Q5: I suspect that the dual-branch design could incur high computational costs (e.g., training time or memory consumption). However, the authors do not provide runtime efficiency with other models. Including such experiments would strengthen the practical relevance of this work.\n\nThank you for your constructive suggestion. We conducted a fair comparison between COME and three other deep models capable of simultaneously addressing both missing views and missing labels under identical experimental protocols, and report their training time and inference time (Unit: seconds) in the following table:\n| Dataset   | Phase     | DICNet   | SIP      | UGDPD-NET | COME (Ours)     |\n|-----------|-----------|----------|----------|-----------|----------|\n| Corel5k   | Training  | 935.65   | 166.17   | 314.34    | 877.03   |\n| Corel5k   | Inference | 0.822    | 0.896    | 0.572     | 0.873    |\n| Pascal07  | Training  | 460.41   | 301.664  | 1480.44   | 1260.50  |\n| Pascal07  | Inference | 5.096    | 5.467    | 3.572     | 4.637    |\n\nAs indicated in the table, COME requires a prolonged training period on both datasets. The additional computational overhead is primarily attributed to the dual-branch architecture in our method. In future work, we plan to explore efficient alternatives to the dual-branch soft pseudo-label generation strategy to optimize computational efficiency.\n> Q6: Why do the authors introduce some methods that can not handle the missing views and labels simultaneously?\n\n Thank you for this important question. Due to the scarcity of methods in iM3C that simultaneously address missing views and labels, we incorporated approaches handling single missing scenarios (e.g., missing views or labels) as comparative benchmarks. Specific details are provided in Appendix C. Moreover, the experimental results demonstrate that models handling individual missing scenarios exhibit suboptimal performance, further underscoring the inherent challenges of iM3C tasks.\n\nReferences:  \n[1] Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models. NeurIPS 2019.\n\nAnonymous link:  \nhttps://anonymous.4open.science/r/6513/rebuttal.pdf", "claims_and_evidence": "The authors claim to learn compact representations by maximizing mutual information, provide rigorous formula derivations, and demonstrate the model's superior performance through experiment results.", "methods_and_evaluation_criteria": "Experimental results demonstrate the model's robustness and superiority across diverse scenarios, with performance evaluated using standard metrics commonly adopted in multi-view multi-label classification tasks.", "theoretical_claims": "The authors hypothesize that classification-relevant information is embedded within the multi-view shared representations. Building on this premise, they propose to learn such shared representations by maximizing cross-view mutual information, accompanied by mathematical derivations.", "experimental_designs_or_analyses": "The authors have presented solid experimental results, including tests under varying missing rates and ablation studies. However, it is essential to clarify whether identical hyperparameters were applied to all datasets. Providing comprehensive experimental details (e.g., dataset-specific hyperparameters) or releasing the source code would significantly enhance the transparency of this work.", "relation_to_broader_scientific_literature": "The paper aims to design a novel network for the incomplete multi-view missing multi-label Classification (iM3C) task, exploring the implementation of compact cross-view representation learning and dual-model pseudo-label generation. On the basis of the existing information theory and hypothesis, the paper explores the problem of contrastive learning in multi-view learning.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Other Strengths:\n1. This paper explores an interesting problem: in practical applications, view missing and label missing are very common, and how to learn the compact semantic information between different views is critical but challenging in multi-view learning.\n2. This article provides a complete explanation of motivation, theoretical derivation and experimental verification.\n3. The proposed compact information learning method is simple but efficient for multi-view classification.\nOther Weaknesses:\n1. Some statements are not clear enough: on Equation (16) and line 625, “Since we adopt the MoE fusion strategy to model $p(z| X^V)$, we have:” which is not very clear.\n2. In the paper, the pseudo label generation models are able to effectively mitigate label missingness. However, beyond the ablation studies, I recommend further investigation into the efficacy of this claim.", "other_comments_or_suggestions": "(Line 88) The period before “(Federici et al., 2020; Tasi et al., 2020)” should be removed. Double check for this manuscript is crucial.", "questions_for_authors": "1.I suspect that the dual-branch design could incur high computational costs (e.g., training time or memory consumption). However, the authors do not provide runtime efficiency with other models. Including such experiments would strengthen the practical relevance of this work.\n2.Why do the authors introduce some methods that can not handle the missing views and labels simultaneously?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "soT2jTRIKd", "reviewer": "Reviewer_oFHQ", "summary": "The authors delve into the study of incomplete Multi-view Missing Multi-Label Classification (iM3C) and aim to address the inadequacy of contrastive learning in dealing with incomplete multi-view data and the negative impact of missing labels. To tackle this problem, they propose a consistent semantic representation learning framework named COME. Firstly, COME learns the minimum sufficient representation by maximizing the mutual information across views. Secondly, it employs a dual-branch soft pseudo-label cross-imputation strategy to mitigate the negative impact of missing supervisory information. To verify the effectiveness of COME, the authors conduct experiments across various datasets and missing settings.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable review. We will address your questions one by one.\n> Q1: Three important hyperparameters in equation 13, $\\lambda_1$, $\\lambda_2$, $\\beta$, need the subsequent experiment to analyze their impact.\n\nThank you for the suggestion. In Appendix D, we investigate the impacts of hyperparameters $\\lambda_1$ and $\\lambda_2$ across three datasets. Experimental results indicate that our method is robust to hyperparameter variations. For visual clarity, Figure 4 in the PDF (provided via the anonymized link) illustrates heatmaps of the average precision (AP) on two datasets. Additionally, in Section 4.3, we analyze the hyperparameter $\\beta$, which balances view-shared and view-specific information. When the $\\beta$ is too small, the model tends to focus excessively on view-specific representations, hindering its ability to learn effective shared representations. Conversely, when $\\beta$ is too large, the model overemphasizes the consistency of shared representations, thereby compressing an excessive amount of information.\n\n> Q2: The authors should check the full text carefully for grammatical errors, such as ‘consitent’ in line 107, “random” in line 116, to meet the high quality requirements of ICML.\n\nThank you for the correction. The typo “consitent” has been revised to “consistent” and “random” in context has been updated to  “randomly” to ensure grammatical accuracy. We thoroughly reviewed the manuscript to detect and fix spelling errors, ensuring compliance with ICML’s high quality requirements.\n\n> Q3: In line 112, “As shown in Fig. 4”, what does that mean? You may want to say “Fig. 1”.\n\nWe apologize for the confusion. We want to illustrate the performance degradation of inadequate contrastive learning by “Fig.1” rather than “Fig.4”.\n\n> Q4: Subfigure needs captions as well, such as that in Fig. 2.\n\nThank you for emphasizing the need for clearer figure captions. In the revised manuscript, we have incorporated detailed captions for each subfigure in Fig. 2.\n\n> Q5: In line 319, “further technical” should be revised to “and further technical”.\n\nThank you for your suggestion. We have revised the phrase “further technical” to “and further technical” in the manuscript as recommended.\n\n> Q6: Why did the author use two separate models for pseudo-tag generation and padding?\n\nSome previous works that train the model using pseudo-labels generated by the same model may lead the model to accumulate error once incorrect pseudo-labels are generated [1,2]. To address this issue, we propose a dual-branch soft pseudo-label generation strategy for missing label imputation. Experimental results on Pascal07 dataset demonstrate that the dual-branch architecture exhibits superior performance and enhanced robustness compared to its single-branch counterpart. As shown in the following table:\n| Model                  | AP     | 1-HL   | 1-RL   | AUC    |\n|------------------------|--------|--------|--------|--------|\n| COME(dual-branch)      | 0.590  | 0.935  | 0.855  | 0.873  |\n| COME(single-branch)    | 0.586  | 0.935  | 0.852  | 0.872  |\n\n> Q7: In Eq. 11, whether the selection of threshold value is empirical or not?\n\nThank you for your question. For the upper bound of the threshold $\\tau_h$, we simply set it to 0.5, which is conventionally used as the threshold in classification tasks. We have conducted analysis experiments to investigate the influence of $\\tau_l$ on five datasets. For pascal07 dataset, we varying the value of $\\tau_l$ from [0, 0.4] with an interval of 0.1. The results are summarized in the following table:\n| Dataset \\ $\\tau_l$  | $\\tau_l=0.4$ | $\\tau_l=0.3 $ | $\\tau_l=0.2$ | $\\tau_l=0.1$ | $\\tau_l=0.0$ |\n|-----------|---------------|---------------|---------------|---------------|---------------|\n| Pascal07  | 0.5852        | 0.5901        | 0.5904        | 0.5873        | 0.5841        |\n\nFrom the results, we simply set the $\\tau_l$ as 0.25 for Pascal07 dataset. Intuitively, during the initial phases of training, a more stringent threshold is employed to ensure the high quality of pseudo-labels, while the threshold is gradually relaxed to extend the range of generated pseudo-labels.\n\nReferences:  \n[1] Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-supervised Multi-label Learning. ECCV 2024.  \n[2] Debiased self-training for semi-supervised learning. NIPS 2022.\n\nAnonymous link:  \nhttps://anonymous.4open.science/r/6513/rebuttal.pdf", "claims_and_evidence": "They claim that a model is proposed to solve the double missing problem of view and label, and experiments confirm the effectiveness of the proposed model.", "methods_and_evaluation_criteria": "The proposed method has obvious pertinence to the iM3C problem, and the metrics used in the paper are also used in many related literatures.", "theoretical_claims": "The authors suppose all the task-relevant information contained by multi-view shared information, which is intuitive and used in previous works.", "experimental_designs_or_analyses": "The experimental design is detailed, and the experiments are conducted separately under different missing rates.", "relation_to_broader_scientific_literature": "This article provides a new approach to solving the problem of insufficient contrastive learning from the perspective of information theory. Based on existing works, it further studies the beneficial effects of mutual information maximization on multi-view learning.", "essential_references_not_discussed": "There are no Not Discussed Essential References have been found.", "other_strengths_and_weaknesses": "Strengths:\n1) The mutual information enhancement strategy effectively compresses redundant information while retaining task-relevant shared semantic information, improving representation learning for incomplete data.\n2)\tThe dual-branch soft pseudo-label generation innovatively reduces the negative impact of missing labels, avoiding error amplification compared to hard pseudo-labeling methods.\n3)\tThe paper is well-written and has high readability.\n\nWeaknesses:\n1) Three important hyperparameters in equation 13, \\lambda_1,  \\lambda_2,   \\beta, need the subsequent experiment to analyze their impact.\n2)\tThe authors should check the full text carefully for grammatical errors, such as ‘consitent’ in line 107, “random” in line 116, to meet the high quality requirements of ICML.\n3)\tIn line 112, “As shown in Fig. 4”, what does that mean? You may want to say “Fig. 1”.\n4)\tSubfigure needs captions as well, such as that in Fig. 2.\n5) In line 319, “further technical” should be revised to “and further technical”.", "other_comments_or_suggestions": "See weakness.", "questions_for_authors": "1) Why did the author use two separate models for pseudo-tag generation and padding?\n2) In Eq. 11, whether the selection of threshold value is empirical or not?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lFtQeWgmfO", "reviewer": "Reviewer_f1Pg", "summary": "The method COME is developed to address the missing data in multi-view multi-label classification tasks, which pursues the maximization of cross-view information to compress the irrelevant information and develops a pseudo-label filling strategy to handle the unavailable labels. Besides, the authors claim that missing data leads to insufficient contrastive learning and then build an information theory-based model to handle it.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your thoughtful and detailed feedback, and we will address your questions one by one.\n> Q1: It would be beneficial for the authors to clarify through experiments why dual-branch model is preferred compared to a single-branch structure.\n\nThank you for your suggestions. We carried out experiments on the Pascal07 dataset to investigate the differences between dual-branch and single-branch architectures. The results are summarized in the following table:\n| Model                  | AP     | 1-HL   | 1-RL   | AUC    |\n|------------------------|--------|--------|--------|--------|\n| COME(dual-branch)      | 0.590  | 0.935  | 0.855  | 0.873  |\n| COME(single-branch)    | 0.586  | 0.935  | 0.852  | 0.872  |\n\nThe experimental results demonstrate that the dual-branch architecture exhibits superior performance compared to its single-branch counterpart.\n\n> Q2: Some details need to be improved, such as the typo “we suppose all the task-relevant information contained by multi-view shared information.” in line 97. I suggest the authors to polish it to help improve the fluency.\n\nThank you for your suggestion. In line 97, the original sentence has been revised to the following in the manuscript: “We suppose that all the task-relevant information is contained in the multi-view shared representation.”.\n\n> Q3: Equations in the Appendix should not appear in the main text commonly, such as Eq. (14).\n\nI am sorry for the serious typos. The corresponding equation is Eq. (6) rather than Eq. (14). We have updated this in the revised  manuscript.\n\n> Q4: Please present the comparative experiments using hard labels and soft pseudo-labels.\n\nThank you for your suggestion. We compared ​the soft pseudo-labeling strategy with ​the hard pseudo-labeling strategy and summarize the results in the follwing table:\n| Model                        | dataset   | AP    | 1-HL  | 1-RL  | AUC   |\n|------------------------------|-----------|-------|-------|-------|-------|\n| COME with hard pseudo-labeling | Corel5k  | 0.425 | 0.988 | 0.916 | 0.918 |\n| COME with hard pseudo-labeling | Pascal07 | 0.585 | 0.934 | 0.852 | 0.872 |\n| COME with soft pseudo-labeling | Corel5k  | 0.432 | 0.988 | 0.917 | 0.920 |\n| COME with soft pseudo-labeling | Pascal07 | 0.590 | 0.935 | 0.854 | 0.873 |\n\nThe experimental results demonstrate that the soft pseudo-labeling strategy exhibits superior performance compared to the hard pseudo-labeling approach.\n\n> Q5: The cross-view reconstruction in Eq.(8) does not use the joint representation z. Could you explain it? Additionally, is the joint representation z utilized in the classification tasks? I think the authors should provide a detailed clarification on this.\n\nThank you for your question. Combining the Mixture of Experts (MOE) with Eq. (15), we derive Eq. (17). This implies that the mutual information between a given view and others can be enhanced through cross-view representation reconstruction, thereby improving semantic consistency. Moreover, the joint representation z has indeed been utilized in multi-label classification tasks, and we have supplemented this in the revised manuscript.\n\n> Q6: In Figure 7, the hyperparameters and appear to tiny influence on the experimental outcomes. Could the authors elaborate on the empirical or theoretical rationale behind this observed insensitivity?\n\nThank you for your question. We conducted sensitivity analysis experiments to investigate the influence of $\\lambda_1$ and $\\lambda_2$ on five datasets by varying their values. Subsequently, we identified hyperparameter configurations that ensured stable model performance and refined the intervals for detailed analysis. These results are visualized in Figure 4 of the PDF document (available via the anonymized link). The experiments show that the model maintains stable performance across a wide range of  $\\lambda_1$ and $\\lambda_2$, demonstrating high robustness to these hyperparameters.\n\n> Q7: The difference in Figure 4(b) is very small. Is this caused by using pseudo labels? How does the model perform at different missing rates if the pseudo-label module is disabled?\n\nThank you for your valuable discussion. We conducted ablation experiments on the pseudo-labeling strategy under varying label missing rates. The performance of COME without this strategy is summarized in Table 1 of the provided PDF document. And the results of COME with the proposed strategy are presented in Table 2 of the PDF. For a clear observation, we show the results in Figure 3 of the PDF. The experimental results demonstrate that the proposed strategy mitigates the negative effects caused by label missing.\n\n> Q8: Figure 3 does not seem to be mentioned in the text. The font styling in Figure 6 is inconsistent.\n\nThanks to the suggestion. The manuscript will be revised to include extended annotations for Figure 3 and to ensure font style consistency in Figure 6.\n\nAnonymous link:  \nhttps://anonymous.4open.science/r/6513/rebuttal.pdf", "claims_and_evidence": "1. Claim: missing data results in insufficient contrastive learning. Evidence: shown in Figure 1.\n2. Claim: all the task-relevant information is contained by multi-view shared information. Evidence: Assumption 2.1 and Proposition 2.2.", "methods_and_evaluation_criteria": "The authors propose a compact semantic learning framework, named COME, for iM3C task. Six metrics such as AUC, AP, OE and so on are used to evaluate the performance of the method.", "theoretical_claims": "Assumption 2.1 and Proposition 2.2 provide the basic theoretical claims for the method, which is solid and reliable according to existing methods.", "experimental_designs_or_analyses": "The authors compared eight methods across five datasets in both incomplete and complete cases. Ablation experiments and parameter analysis experiments were conducted. It would be beneficial for the authors to clarify through experiments why dual-branch model is preferred compared to a single-branch structure.", "relation_to_broader_scientific_literature": "In contrast to prior studies, the manuscript explores how contrastive learning degenerates in multi-view multi-label classification when both view and label-missing coexist. By learning compact representations, the degeneration issue is alleviated, thus enabling contrastive learning to remain robust even in the case of view missing. The authors proposed a novel dual-branch architecture to generate soft pseudo-labels, effectively addressing the label missing problem.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "**Strengths**\n1. Understanding the performance degradation mechanisms of contrastive learning under incomplete multi-view scenarios remains an open challenge, with significant implications for robust representation learning. The authors proposed an effective solution to this problem and I think this framework has the potential to be extended to other multi-view learning tasks.\n2. Overall, the experimental results presented in this paper are rich and convincing.\n3. The description of the method is clear and understandable, which I think will be helpful for readers to replicate the results.\n4. The authors study the complexity of the multi-view multi-label classification problem from various aspects, including multi-view representation learning and multi-label classification, from feature extraction to pseudo-label generation, showing a large workload.\n\n**Weaknesses**\n1. Some details need to be improved, such as the typo “we suppose all the task-relevant information contained by multi-view shared information.” in line 97. I suggest the authors to polish it to help improve the fluency.  \n2. Equations in the Appendix should not appear in the main text commonly, such as Eq. (14).\n3. The authors claim that in pseudo-label generation, the use of hard pseudo-labels may lead to error accumulation and over-fitting, but there is no relevant comparative experiment in the paper. Please present the comparative experiments using hard labels and soft pseudo-labels.", "other_comments_or_suggestions": "1. The cross-view reconstruction in Eq.(8) does not use the joint representation z. Could you explain it? Additionally, is the joint representation z utilized in the classification tasks? I think the authors should provide a detailed clarification on this.\n2. In Figure 7, the hyperparameters $\\lambda_1$ and $\\lambda_2$ appear to tiny influence on the experimental outcomes. Could the authors elaborate on the empirical or theoretical rationale behind this observed insensitivity?", "questions_for_authors": "1. The difference in Figure 4(b) is very small. Is this caused by using pseudo labels? How does the model perform at different missing rates if the pseudo-label module is disabled?\n2. Figure 3 does not seem to be mentioned in the text. The font styling in Figure 6 is inconsistent.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MzvUvQfJe3", "reviewer": "Reviewer_8PHw", "summary": "The paper develops a model named COME for incomplete multi-view missing multi-label classification tasks. Unlike most existing methods, the approach aims to learn compact semantic information by minimizing task-independent redundant information. Additionally, a dual-branch soft pseudo-label generation strategy is introduced in the model to alleviate the negative impact of missing supervisory information.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive reviews and suggestions. Below, we will address each of your questions.\n> Q1: Some minor mistakes should be revised, the reference of equation 14 in line 205 is incorrect and it should be capitalized in line 374: “In table 1”.\n\nThanks for your correction. We have corrected the incorrect reference to Eq. (14) in line 205, capitalized “Table 1” in line 374, and addressed other minor errors throughout the manuscript to ensure consistency with academic standards.\n\n> Q2: The code is not provided. To improve reproducibility, implementation details and code should be provided.\n\nThanks for your suggestion. The code will be made publicly available upon acceptance to support reproducibility.\n\n> Q3: I noticed that in the experimental results on the complete dataset in Figure 8, there are only results of UPDGD on the Corel5k and Pascal07 datasets. Could you provide more results on other datasets?\n\nWe sincerely apologize for this and we have provided the complete experimental results on the five datasets without any missing views and labels in Figure 1 of the PDF document (provided via the anonymized link). It can be observed that our Model (COME) achieves superior performance across six metrics compared to other methods on most datasets.\n\n> Q4: Is the data still incomplete in the inference phase? I know that views and labels are not complete in the training, so is this still the case when inferencing?\n\nWe still maintain the view’s incompleteness in the inference phase, but we do not use any missing settings for labels as the evaluation criteria for real performance.\n\n> Q5: In Figure 2(b), is this label distribution common? Are the patterns found in a single dataset universal?\n\nThanks to your questions. The label distribution of the other datasets is summarized in Figure 2 of the PDF document. We observed that the number of labels per sample varies across datasets and within individual datasets. This inherent variability makes the Top-K pseudo-labeling strategy [1] unsuitable for such scenarios.\n\n> Q6: From Figure 4, why is the negative impact of missing labels smaller than that of missing views? More discussions should be added.\n\nThank you for your discussion. Intuitively, in the fusion stage of multi-view representations using a Mixture of Experts (MoE), the learned weights reflect the relative importance of different view representations within the fused representation. The absence of key view representations causes rapid degradation in the quality of multi-view joint representations, which poses significant challenges to multi-label classifiers. Additionally, we introduce a dual-branch soft pseudo-label imputation strategy to mitigate the multi-label missing problem. We present the ablation results of the “dual-branch soft pseudo-label imputation” strategy under varying missing label rates.  The results of COME without dual-branch soft pseudo-label imputation strategy are summarized in the following table:\n| label-missing rate | AP    | 1-HL  | 1-RL  | AUC   |\n|--------------------|-------|-------|-------|-------|\n| 0%                | 0.604 | 0.937 | 0.862 | 0.879 |\n| 30%               | 0.594 | 0.935 | 0.857 | 0.875 |\n| 50%               | 0.586 | 0.935 | 0.854 | 0.872 |\n| 70%               | 0.571 | 0.934 | 0.843 | 0.864 |\n\nAnd the results of COME with dual-branch soft pseudo-label imputation strategy are summarized in the following table:\n| label-missing rate | AP    | 1-HL  | 1-RL  | AUC   |\n|--------------------|-------|-------|-------|-------|\n| 0%                | 0.602 | 0.937 | 0.859 | 0.878 |\n| 30%               | 0.596 | 0.936 | 0.859 | 0.876 |\n| 50%               | 0.590 | 0.935 | 0.855 | 0.873 |\n| 70%               | 0.580 | 0.933 | 0.852 | 0.870 |\n\nFor clarity, the results are presented in Figure 3 of the PDF. The experimental results demonstrate that the proposed strategy effectively mitigates the negative impact of missing labels, particularly under a high missing rate of 70%.\n\nReferences:  \n[1] Class-Distribution-Aware Pseudo-Labeling for Semi-Supervised Multi-Label Learning. NeurIPS 2023.\n\nAnonymous link:  \nhttps://anonymous.4open.science/r/6513/rebuttal.pdf", "claims_and_evidence": "The claims are clear and convincing. Main claim: the proposed multi-view semantic consistency enhancement strategy learns compact multi-view shared information, which effectively alleviates the performance degradation caused by incomplete contrastive learning.", "methods_and_evaluation_criteria": "The authors used six evaluation metrics commonly used in multi-label classification, which are rational and common.", "theoretical_claims": "The adequacy hypothesis of multiple views shared information proposed in the paper is reasonable.", "experimental_designs_or_analyses": "A large number of experiments are carried out on five datasets. The experimental settings are reasonable, and the experimental analysis is sufficient.", "relation_to_broader_scientific_literature": "This paper studies the inadequacy of contrastive learning in dealing with incomplete multi-view data, which is worth further exploration. Unlike most existing methods, the proposed method aims to learn compact semantic information by minimizing task-independent redundant information.", "essential_references_not_discussed": "The literature citations of the article are reasonable, and the latest relevant theories and specific methods are introduced in detail.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is well-written, with explicit problem definition and clear description. It’s readily understandable even without prior knowledge.\n2. The paper provides comprehensive experimental results to validate the effectiveness and robustness of the proposed methods.\n3. The setting of double incompleteness in views and labels is novel, and the proposed pseudo-label strategy alleviates the problem of missing labels.\n\nWeaknesses:\n1. Some minor mistakes should be revised, the reference of equation 14 in line 205 is incorrect and it should be capitalized in line 374: “In table 1”.\n2. The code is not provided. To improve reproducibility, implementation details and code should be provided.", "other_comments_or_suggestions": "I noticed that in the experimental results on the complete dataset in Figure 8, there are only results of UPDGD on the Corel5k and Pascal07 datasets. Could you provide more results on other datasets?", "questions_for_authors": "1.\tIs the data still incomplete in the inference phase? I know that views and labels are not complete in the training, so is this still the case when inferencing?\n2.\tIn Figure 2(b), is this label distribution common? Are the patterns found in a single dataset universal?\n3.\tFrom Figure 4, why is the negative impact of missing labels smaller than that of missing views? More discussions should be added.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper received four effective reviews, and all of them are positive. Overall, the paper is of good quality and should be accepted.
Decision: Accept (poster)