Robust Contrastive Multi-view Clustering against Dual Noisy Correspondence
Authors: Ruiming Guo
Mouxing Yang
Yijie Lin
Xi Peng
Peng Hu
Venue: NeurIPS 2024 poster (2024)
Keywords: multi-view clustering; noisy correspondence
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Recently, contrastive multi-view clustering (MvC) has emerged as a promising avenue for analyzing data from heterogeneous sources, typically leveraging the off-the-shelf instances as positives and randomly sampled ones as negatives. In practice, however, this paradigm would unavoidably suffer from the Dual Noisy Correspondence (DNC) problem, where noise compromises the constructions of both positive and negative pairs.
Specifically, the complexity of data collection and transmission might mistake some unassociated pairs as positive (namely, false positive correspondence), while the intrinsic one-to-many contrast nature of contrastive MvC would sample some intra-cluster samples as negative (namely, false negative correspondence).
To handle this daunting problem, we propose a novel method, dubbed Contextually-spectral based correspondence refinery (CANDY). 
CANDY dexterously exploits inter-view similarities as \textit{context} to uncover false negatives. Furthermore, it employs a spectral-based module to denoise correspondence, alleviating the negative influence of false positives. 
Extensive experiments on five widely-used multi-view benchmarks, in comparison with eight competitive multi-view clustering methods, verify the effectiveness of our method in addressing the DNC problem.
The code is available at https://github.com/XLearning-SCU/2024-NeurIPS-CANDY.

Robust Contrastive Multi-view Clustering against Dual Noisy Correspondence
Ruiming Guo Mouxing Yang", Yijie Lin', Xi Pengl Peng Hult College of Computer Science, Sichuan University, China
State Key Laboratory of Hydraulics and Mountain River Engineering. Sichuan University, China
(guoruiming gm, yangmouxing, linyijie gm, pengx. gm, penghu venghu.ml>Agmaal @gmamm com
Abstract
Recently, contrastive multi-view clustering (MvC) has emerged promising avenue for analyzing data from heterogeneous sources, typically leveraging the off-the-shelf instances positives and randomly sampled ones negatives. In practice, however, this paradigm would unavoidably suffer from the Dual Noisy Correspondence (DNC) problem, where noise compromises the constructions both positive and negative pairs. Specifically, the complexity of data collec- tion and transmission might mistake some unassociated pairs as positive (namely, false positive correspondence), while the intrinsic one-to-many contrast nature of contrastive MvC would sample some intra-cluster samples negative (namely, false negative correspondence). To handle this daunting problem, we propose novel method, dubbed Contextually-specttr based correspondence refinery (CANDY). CANDY dexterously exploits inter-view similarities as context to uncover false negatives. Furthermore, it employs a spectral-basee module denoise correspondence, alleviating the negative influence of false positives. Ex- tensive experiments on five widely-used multi-view benchmarks, in compari- son with eight competitive multi-view clustering methods, verify the effective- ness of our method in addressing the DNC problem. The code available bttts://githue com
1 Introduction
In real-world applications, data are often presented in various modalities views, including but not limited to visible images, thermal images, text, and audio [ 2. Multi-view Clustering (MvC), fundamental tool in multi-view data analysis, aimed at learning common space in which data are grouped into distinct clusters, attracts significant attention across various research communities (349). In recent years, contrastive MvC methods have emerged central focus in multi-view clustering researches [1] The typical implementation these methods involves leveraging the off-the-shelf data pairs as positives and randomly sampling cross-view pairs as negatives, followed by employing contrastive learning upon them 12 As result, the cross-view discrepancy could be eliminated, revealing the underlying cluster structure.
Although existing contrastive MyC methods have achieved promising performance, their success heavily relies on the assumption of faultless cross-view correspondence. In practice, however, this assumption hard or even impossible meet [15] leading inevitable contamination of the cross-view correspondence, as shown in Fig. Tb. More specifically, the complexity of data collection
"qqual contribution Corresponning author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

vi
Fake Negative
Falee Negative
14
Similar
Disimilar
Nalun Negative
True
Positive
Cuntest-hused Symantic Mining
1
22
View
realiss
SVD
Falur Positive
Positive
v4
14
5094
254
5064
12:42
Similar
Dissiiilar
Spectral-basee based Correspondence Denobinn
(a) Dual Noisy Correspondence
(b) Our key idea
Figure 1: The motivation and key idea. (a) Dual noisy correspondence. The cross-view data pairs are contaminated by both false positive and negative correspondences, and the clean and noisy correspondence is mixed. (b) Top: Context-based Semantic Mining. The existing studies estimate the data affinity based on the data representation and might neglect the out-of-neighborhood yet semantically- assscciaeed false negatives. In contrast, formulate the affinity from data point to all the others as the context and use them for similarity induction, thus benefiting the false negative uncovering in global manner; Bottom: Spectral -based Correspondence Denoising. Borrowing from spectral decomposition for signal denoising, we employ spectral denoising on the contextual affinity graph to prevent false positives from dominating the model optimization. In the figure, the thickness of the black arrows represents the association strength between two data points.
and transmission might mistake certain unaligned cross-view pairs for positive pairs, leading to false positive correspondence. Conversely, the inherent one-to-many contrast characteristic of contrastive MvC would inevitably result in lemantically-associan as negatives, thus producing false negative correspondence.
cross-view positives being wrongly treated
Based on the above observations, this paper reveals a novel problem for contrastive MvC called Dual Noisy Correspondence (DNC). Formally, DNC refers to the noise present in both cross-view positive and negative pairs. This problem is akin to the partially view-aligned problem (PVP), yet differs in that PVP presupposes the availability of some correcty--sssociate instances for training, while DNC breaks through this impractical assumption and remains agnostic to any clean correspondence 20 21).
Thus, DNC could be regarded as more practical yet challenging variant of PVP, resulting in the infeasibility of PVP-oriented methods to address the DNC problem. Notably, our experimental findings, detailed in Section 4 support this claim.
To tackle the DNC problem. we present a novel robust method, dubbed ContextuAlll--ppeet based correspoNDence refinerY (CANDY), for learning to cluster with noisy positive and negative correspondences. As illustrated in Fig. Tb. CANDY consists of two core modules: i) the Context- based Semantic Mining (CSM) module for recalling the false negatives, and ii) the Spectral-based Correspondence Denoising (SCD) module for alleviating the adverse impact of false positives. To be specific, CANDY first constructs cross view affinity graph from the multi-view data. Subsequently, CANDY calculates the connection probabilities from each node to all others, forming the context, and exploits CSM to induce a high-order contextual affinity graph. Thanks to the properties of high-order affinity, CSM could facilitate the discovery of semantically-associn in the negatives. After that, inspired by singular value decomposition techniques used in image denoising [22) 23), CANDY performs spectral decomposition on the contextual affinity graph and employs SCD filter noise the graphical spectrum, thus mitigating overfitting to false positives. Finally, CANDY employs the denoised contextual affinities to weight arbitrary contrastive losses to achieve robust MvC against DNC.
positives hidden
In summary, the main contributions and novelties of this work could be summarized as follows.
We reveal and study a new practical problem in contrastive multi vieww clustering, namely, dual noisy correspondence (DNC). Unlike prior PVP-oriented studies that rely on quite few correctly-associater pairs, DNC refers to noise inherent in both cross-view positive and negative pairs. To the best of our knowledge, this could be one of the first investigations into noisy correspondence within MvC, particularly the more practical and challenging DNC problem.

---

We propose a novel robust method called CANDY for enhancing the robustness of contrastive MvC against DNC, embracing the following novelties: The formulation of affinity from one data point to others as context, facilitating the revelation of false negatives; and ii) Spectral denoising upon the high-order affinity graph, preventing overfitting to false positives.
Extensive experiments verify the effecttiveness and superiority of CANDY. Moreover, demonstrate the generalizability of CANDY, showing that it could serve as a plug-and-pls solution to enhance the robustness of most contrastive MvC methods against DNC.
2
Related Work
In this section, present brief review of two topics related to this work: multi- clustering and noisy correspondence learning.
2.1 Contrastive Multi-view Clustering
The inherent pairing characteristic of the multi-view data renders the contrastive learning paradigm natural fit for MvC, giving rise to the established paradigm of contrastive MvC. Existing MvC methods could roughly classified into the following three groups: i) Vanilla contrastive MvC methods (24). which directly exploit contrastive learning to enhance the discrimination of learned representations by maximizing the mutual information between distinct views. ii) Robust contrastive MvC methods against incomplete instances (75) which employ contrastive learning to learn the cross-view consistency, thereby facilitating the recovery of missing samples. iii) Robust contrastive
MvC methods against false negatives [8] 28) which redesign dedicated loss functions or similarity estimation techniques to conquer false negatives inherent in contrastive learning, thus boosting clustering performance.
Our CANDY. alongside the works of [8] 28 is devoted to addressing false negatives, while having the following significant distinctions. Different from [18] which utilizes false-negaaive-rroust loss, CANDY presents a Context-based Semantic Mining (CSM) module to induce a context-aware and high-order affinity graph, benefitnng the discovery of false negatives from a global perspective. Moreover, (28) proposes modeling the probability of false negatives by resorting to random walks while being susceptible to cross-view false positives. In contrast, thanks to the SCM module, CANDY embraces a more robust performance in uncovering false negatives, as verified in our experimental results.
2.2 Noisy Correspondence Learning
In the era of big data, millions of multimodal data are crawled from the Internet, often requiring extensive curation, which time-intensive and prst-prohibitive [15]17]. Nevertheless, it is almost impossible to eliminate misaligmment in a large quantity of multimodal data, leading to noisy correspondence. To handle this problem, noisy correspondence learning is presented to alleviate the negative influence of false positive and negative correspondences within data pairs, which has achieved promising results across various applications, such as cross-modal retrieval [1) object re-identification (34), multi-view learning [21 35), graph matching [36), video reasoning [37] image-text pre-training [38].
To the best of our knowledge, this work could be one of the first studies on learning to cluster with noisy correspondence. Unlike most existing approaches focusing solely on either false positives or negatives [30] 21). our CANDY addresses the more general challenge called Dual Noisy Correspon- dence (DNC). Extensive experiments reveal the impracticality of applying the existing approaches to DNC in MvC, highlighting the necessity of a tailored solution to MvC against the DNC problem.
3 Method
In this section, we elaborate on the proposed ContextuAAll--speerra based correspoNDence refinerY (CANDY), which aims to enhance the robustness of contrastive MvC against the Dual Noisy Cor- respondence (DNC) problem. As illustrated in Fig, our CANDY consists of two novel modules:

---

Context-based Semantic Mining
Spectral-basse Correspondence Denoising
2
False Negathe Disimilar
X
2
x!il
Similar
2
SVD
2
False Negalive Similar
Falee Positive
Fulse Posilive
Similor
Disimilar
Caussia Kernel
Similarity
Contest as Representation
High-order Affinity Gnaph
DNC-robust Pscudo Target
Contrastive Learning
Figure 2: Overview of CANDY. First, each view is fed into view-speciff encoder generate the embeddings. These embeddings are adopted to construct both inter- and intra-view affinity graphs, with edges weighted by Gaussian kernel similarity. The context-based semantic mining module dexterously reformulates inter-view similarities "context", employing this context set of bases to induce a new contextual affinity space. In this space, the rooted/dissimilar false negatives could be brought to light. Second, the spectral based correspondence denoising module steps in alleviate the adverse impacts noisy correspondence on positive pairs, thus obtaining a low-noise pscudo target. Finally, this pseudo target steers the contrastive learning process, enhancing robustness against DNC in MvC. For the sake brevity, this figure only presents simplified depiction involving two views, and the robust contrastive MvC from view to view 2.
context-baue based semantic mining module to uncover inherent false negatives, and a spectral- based correspondence denoising module to prevent contrastive MvC from overfitting false positives. In the following, we commence with the mathematical formulation of the DNC problem in Section 3.1 proceed the context- based semantic mining module Section 3.2 and culminate with the spectral- based correspondence denoising module in Section 3.3
3.1 Problem Formulation
{(x ......,, x; (V) D with instances observed from V. views, the objective of contrastive MvC is to group these instances into K clusters. To this end, contrastive MvC methods construct the of posi- tive and negative pairs U ((x()) (wz) .e U 199
Given the multi-view dataset
(1)
1a= 1,1 01 V. * 02 and .c.) |4 1c=00,15 01, V.u. # 02 by utilizing the off- tre-the-shell in-
(vi)
{ (1)) (od))
stances and perform random sampling across views respectively. where denotes the established cross-view correspondence. Subsequently, the contrastive loss 33 40) is applied to eliminate the cross-view discrepancy and reveal the cluster structure. However, as elaborated in the Introduction, cross-view correspondence could often be contaminated by both false positives and negatives. More specifically, a certain amount of unassociated (é 0) and associated treated as positives (c 1) and negatives (c 0) respectively, while the ground-trutt correspondence is unknown. particular, the ratio of false negatives would reach up to 1/k when the categories of the dataset D are uniformly distributed, where à is the number of classes.
1) pairs would be wrongly
To counter the DNC challenge, we introduce soft contrastive loss:
Σ H (C(m..))) (Z(11))((a))
(1)
#1=1
=1=1
where H denotes the row-wise cross-entropy function with mean reduction, (( E R
the pseudo target (Eq. 6). (1)) ((2)) represents the affinity matrix between views U1 and U2 and (-) signifies the softmax function. The batch-wise representation matrix (v) E Baxd encapsulates features extracted by the vpewispecific encoder f(1), with denoting the batch size. The softmax

---

function (p(+)) is applied row-wise to ensure each row sums to one as follows:
[Z(v
Z (12) /T)
(z((n))z(n) )1,
exp
(2)
E exp [Z]] [Z (v2)), /T)
In general, traditional contrastive MvC methods assume that the gross-view correspondence is faultless, typically adopting an identity matrix I E x="x" as the target. As verified in our experiments, such a vamilla target not only misleads the model overfit false positives but also neglects numerous semantically associated false negatives. Therefore, the goal of CANDY becomes generating a robust pseudo target resilient against the DNC problem.
3.2
Context-based Semantic Mining
The crux of uncovering false negatives lies in accurately modeling the semantic association between data points. Therefore, the widely-used strategy based on the point-to-point similarity in the affinity graph. Specifically, a fully-connected affinity graph A is first constructed using the feature ((r) and Z (*2) as nodes in a mini-batch, with edge weights defined by Gaussian kernel similarity. Mathematically,
(o) (01)) exp (-1|2-00, [1),
|
|z (z))
/0
(3)
where is scale parameter and i1 is the anchor view. After that, where each edge represents the probability of semantic association between the corresponding two nodes, could be obtained by normalizing in row-wise manner. This strategy, however, tends to be short-sighted, potentially neglecting the out-of-neighborhood yet semantically-asociia false negatives, shown in Fig. T and supported by our experiments.
view graph ( (v2))
In contrast, a simple yet effective semantic modeling strategy is presented to formulate the connection probability from one node to all others context, thereby redefining the context special
representation for semantic mining. Intuitively, the context A (e) -17)) | (e)+t1)) A (01 [x))
in
serves as a new embedding for the node i, facilitating the construction of a cross-view high-order affinity graph (*))))2) follows:
G (1i = A (0) +22) A (12-+02)
Thanks to context modeling, our CSM embraces two distinct advantages: i) it encapsulates the structural information of nodes into the graph, enhancing the ability global semantic mining, and ii) it provides novel basis for data representation project nodes into new affinity space, potentially better uncovering semanticall--assoiiate false negatives.
3.3 Spectral-based Correspondence Denoising
The false positive correspondence would emerge in both the off-the-sheef positive pairs, as elaborated in Section]] [ and the wrongly associated negatives during the construction of G (01-+14) To address this, we propose a correspondence-dentte
mechanism for the high-order affinity graph G ((1)) based the spectral denoising theorems [22] [33. In brief, it is widely acknowledged that the eigenvectors of signals corresponding to larger eigenvalues represent principal components, while smaller ones are apt be noise. By selectively discarding the information tied to the minor eigen- values, one could filter out the noise, thereby revealing the underlying structures. Inspired by these preliminary insights, we propose refining (w by resorting to singular value decomposition. Mathematically.
G
UEVT.
(5)
where Σ denotes a diagonal matrix consisting of the singular values, U and V is the left singular matrix and the right singular matrix, respectively.
After that, the denoised pseudo target could be obtained via
G (o) -22) U.VT.
(6)
5

---

where Σ = diag(>i," (L) is diagonal matrix consisting of the retained singular values (A1 > A,. 2 17). with 7) being a denoising hyper-paramett fixed as 0.2 our experiments.
By combining the denoised pseudo target with the vanilla target O we obtain the noise-resisted pseudo target (C)))))) for the proposed soft contrastive loss (Eq.) via
(trr.e)) AI ((n)))))
where À is fixed as 0.2 in our experiments.
4 Experiments
In this section, we verify the effectiveness of our CANDY against the DNC problem through extensive experiments by addressing the following questions:
1. Performance Superiority:
Does CANDY outperform the existing ofate-of-the-art (SOTA) MvC methods, including those designed for PVP?
2. Component Indispensability Are all components crucial for maintaining robustness against DNC?
3. Working Mechanism:
How does CANDY achieve robustness against DNC?
4. Approach Necessity: instead of using existing noisy correspondence learning methods?
Why is it necessary design approach for the DNC problem
5. Approach Generalizability Can CANDY be used in a plug-and and-play manner to endow other contrastive MvC methods with robustness against DNC?
4.1 Configurations and Implementation Details
CANDY is designed as a plug-and-plaz and-play solution to endow most existing contrastive MvC methods with robustness against the DNC problem. Therefore, we choose the SOTA contrastive MvC method, namely, DIVIDE [28] as our baseline. Specifically, we retain the architecture and pipeline of DIVIDE, modifying only the loss function. Following DIVIDE, to obtain good initialization for the neural networks, we use the vanilla contrastive loss by setting the target ((1).2)) in Eq. 1 as the identity matrix I for the first 20 epochs of training. To endow DIVIDE with robustness against DNC, we incorporate context-basse alongside the soft contrastive loss (Eq. [ Since MvC requires training and clustering on the same dataset, we conduct the view realignment strategy on the learned representation by following the PVP studies [0 21 For achieving clustering. we concatenate the realigned representations across views to form a common representation of the MvC data and then apply the k-means algorithm by following 25
semantic mining and peectral-based correspondence denoising modules,
In the experiment, CANDY is implemented with PyTorch 2.1.2, and the model is optimized with the Adam 41 optimizer with a learning rate of 0.002 across all experiments, with a batch size fixed to 1024. All evaluations are conducted on Ubuntu 20.04 OS with NVIDIA 3090 GPUs. The scale parameter o in Eq. Bs fixed 0.07 across all experiments. The experiments are carried out on the following five widely-used multi-vies view learning datasets.
Scene-15 42 includes 4,485 images across 15 categories. We employ PHOG and GIST as two distinct views following [8].
Caltech-101 43
consists 8,677 images collected from 101 classes. We use two kinds of deep features extracted by the DECAF and VGG19 neural networks as two views
following [44]
LandUss-21
contains 2,100 satellite imagery samples in 21 categories. We employ the PHOG and LBP features as two views following Lin et al. [46).
45
is repository of news content in multiple languages with 18,758 samples. Following 48 we transform the texts into a 10-dimensional latent space with a conventional autoencoder and use English and French as two different views.
Reuters
47

---

Table 1: The statistics of false positive and false negative ratios (%) with respective to different datasets and in the experiments.
1a
Lsnd
NUSWIDE
Reuters
Scene15
n
FP
FN
FP
FN
FP
FN
FP
FN
FP
FN
0.0
0.00
2.84
0.00
4.73
0.00
9.99
0.00
21.40
0.00
6.91
0.2
19.34
2.84
19.10
4.73
17.98
9.99
15.67
21.40
18.68
6.91
0.5
48.45
2.84
47.33
4.73
45.07
9.99
39.64
21.40
46.42
6.91
0.8
77.48
2.84
76.24
4.73
72.03
9.99
62.57
21.40
73.98
6.91
NUS-WIDE [49] includes 9,000 images paired with their respective captions from 10
classes. We adopt a VGGI9 neural network for the extraction of visual features, and Sentence CNN to extract the text features by following [50]
For comprehensive evaluations, we vary the noise ratio in the datasets by adopting the following protocols. For the false positive correspondence, we select one view as the anchor and randomly shuffle samples in other views according to the specified FP ratio 1) which is varied from 0%. 20%, 50%, 80%. For the false negative correspondence, we adhere to the inherent FN ratio in each dataset. For clarity, we present the statistics of FP and FN ratios for different datasets in Table [ Notably, as the samples within the same instance would be regarded as negative if they do not belong to the same class, the practical FP ratios might be slightly lower than the specified 77.
4.2 Comparison with State of the Arts (Performance Superiority)
In this section, compare CANDY with eight SOTA MyC methods including the typical MyC
methods (DCCAE [TI, BMVC (52), the PVP-oriented MvC methods (MvCLN [21]. PVC [20),
SURE [18) CGCN (33), the selti-negative-rouust contrastive MvC (GCFAgg [54] and DI-
VIDE (8)). Following the widely-used evaluation protocols, we adopt "ACC", "NMI" and "ARI" as the metries.
Table presents the comparison results for each dataset and the average results overall, where one could have the following observations. First, our CANDY outperforms all baselines in terms of the average ACC and ARI when the FP ratio is 0%. which could be attributed to the powerful semantic mining capacity on the false negatives, Second, all baselines experience heavy performance degradation when encountering false positives. In contrast, CANDY achieves significant robustness and remarkably outperforms all baselines by a large margin. The above two observations could verify the effectiveness of CANDY against the DNC problem.
Furthermore, we explore the capacity of CANDY on handling the other important problem in MvC, namely, missing views. To this end, we follow DIVIDE 28 recover the missing views. We conduct experiments on four widely-used incomplete MvC benchmarks and compare CANDY with other baseline methods [2] 55457 I8] 58-600 28). As demonstrated in Table CANDY could achieve competitive results comparable to SOTA methods, even though it is primarily designed for handing DNC rather than missing modalities.
4.3 Ablation Studies and Parameter Analysis (Component Indispensability)
In this section, we conduct ablation studies and parameter analysis to investigate the indispensable role and robustness of our modules.
As shown in Table we design the following four method variants for the ablation studies: Warmup Only: using the identity matrix as the target for Eq. [hhoughouu the training process; ii) Re-alignment: adopting re-alignment strategy like the PVP studies; iii) SCD: performing the SCD module to denoise the vanilla affinity graph A (v1-)))) and using the resulting graph as the target for Eq. iv) CSM: the complete version of CANDY, adopting the CSM module to induce ((1)- for recalling the false negatives and performing Eq. to obtain the final pseudo target. From the results, one could observe that both the SCD and CSM modules play important roles in achieving robustness against DNC.
7

---

Table 2: Clustering performance comparisons on five widely-used multi-view datasets. The results the mean of five individual runs. The best and second best results shown in bold and underline, respectively.
FP Ratio
Methods
Scene15
Caltech-100 101
LandUse21
Reuters
NUS-WIDE
Average
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
ACC
NMI
ARI
DCCAE (ICME'1S)
34.6
39.0
19.7
45.8
68.6
37.7
15.6
24.4
4,4
42.0
20.3
8.5
47.5
17.1
37.6
37.1
33,9
21.6
BMVC BMVC.(TPMM116
40.5
41.2
24.1
50.1
72.4
33.9
25.3
28.6
11.4
42.4
21.9
15.1
36.0
21.0
16.5
38.9
37.0
20.2
PVC (NeurIPS 20)
38.0
39.8
21.1
20.5
514
15.7
16.8
25.2
5.6
44.1
27.1
27.1
19.3
777
3.8
27.7
30.2
14.7
MVCLN (CVPR'21)
37.9
42.3
25.6
39.6
65.3
32.8
26.1
30.7
12.5
38.8
42.1
25.2
54.1
38.3
35.7
39.3
43.7
26.4
SURE (TPAM)
43.8
70.1
25.1
10.9
49.1
23.6
38.1
43.3
25.5
GCFAgg (CVPR
42.2
42.5
24.4
56.6
80.7
37.9
27.5
31.3
14.0
34,4
23.8
10.5
41.1
32.1
18.6
40.4
42.1
21.1
CGCN (TCSVT'24)
42.9
43.4
25.0
49.1
75.2
33,8
28.8
36.0
15.0
45.8
27.0
22.3
61.2
48.1
41.2
45.6
45.9
27.5
DIVIDE (A4AI'24)
49.1
48.7
31.6
62.2
83.0
50.5
32.3
39.7
18.1
59.3
39.5
29.0
45.1
30.9
19.4
49.6
48.4
29.7
CANDY (Ours)
42.0
41.6
24.7
67.3
83.8
60.0
30.6
36.5
16.2
57.7
30.8
37.1
62.1
49.0
37.0
51.9
48.3
35.0
DOCAE (ICMI'15)
32.9
17.1
29.6
36.9
39.2
60.1
15.0
3.8
17.4
41.6
13.1
19.3
41.6
11.6
26.9
33.6
17.0
30.7
BMVC (IPAMI'18)
20.0
10.2
4.7
42.7
58.2
24.6
16.1
13.0
4.3
36.4
11.9
8.1
27.7
10.7
7.7
28.6
20.8
9.9
(NeurIPS
32.4
13.6
24.3
MVCLN (C)PR'21)
39.3
36.7
21.7
43.3
64.0
52.8
24.4
26.1
10.8
37.9
35.9
20.3
42.5
29.3
21.3
37.5
38.4
25.4
20%
SURE (TPAMT23)
40.0
37.3
21.5
26.9
49.9
18.0
25.2
27.4
11.6
40.7
20.9
15.8
57.0
45.0
38.6
38.0
36.1
21.1
GCFAgg (CVPR 23)
40.9
38.6
22.7
50.1
70.6
30.1
25.7
27.8
11.9
35.2
19.0
10.8
38.6
23.3
15.6
38.1
35.9
18.2
CGCN (TCSVT'24)
40.7
38.0
22.1
40.8
64.9
27.2
27.0
31.4
13.3
43.5
23.0
19.4
58.0
41.7
35.9
42.0
39.8
23.6
DIVIDE (AAAI'24)
39.9
30,9
16.2
55.3
31.0
41,9
25.6
CANDY (Ours)
40.4
40.3
23.7
65.9
82.3
60.1
30.5
35.3
15.7
54.2
27.9
33.8
60.3
47.1
36,9
50.3
46.6
34,0
DOCAE (ICML'15)
26.8
10.2
27.0
26.8
13.3
13.2
37.7
12.5
32.3
13.5
11.2
21.8
BMVC (18AML'18)
13.6
3.9
26.5
34.2
8.9
13.5
7.5
1.9
26.6
3.3
2.3
18.4
3.1
1.9
19.7
10.4
3.3
PVC (NeurIPS 20)
20.3
10.2
13.6
7.4
21.8
5.0
20.6
28.5
8.7
42.9
23.5
23.4
24.1
10.1
9.9
23.1
18.8
12.1
MVCLN (CVPR'21)
41.3
19.7
15.1
214
39
11.7
21.4
21.8
7.8
34.8
35.5
19.7
31.7
16.6
10.7
30.1
26.5
13.0
50%
SURE (TPAMP23)
35.7
19.9
41.7
38.0
14.3
35.0
17.
12.0
27.2
13.7
GCFAgg (CVPR'23)
32.9
42.2
63.0
25.2
10.9
28.5
4.5
26.7
10.5
6.4
31.3
28.0
12.8
CGCN (ICSVT'24)
32.5
29.5
15.7
33.4
59.3
21.6
25.8
28.2
11.9
40.5
16.1
14.1
50.1
33.8
27.4
36.5
33,4
18.1
DIVIDE (44AA'24)
37.4
34.0
20.3
39.1
58.7
32.5
28.1
30.4
13.5
41.2
19.4
14.8
44.0
23.9
16.6
38.0
33.3
19.5
CANDY (Ours)
41.3
39.4
24.0
60.7
79.0
56.6
29.9
33.1
15.2
47.4
21.7
27.3
58.1
43.2
34.5
47.5
43.3
31.5
DOCAE (ICML'15)
20.9
6.7
14.4
18.4
15.8
41.8
14.5
3.2
13.4
35.3
7.6
10.0
36.2
14.9
21.9
25.1
9.6
20.3
BMVC (IPAMI'18)
10.5
1.5
0.3
11.9
18.3
10.1
4.2
21.3
0.1
13.1
0.6
0.2
13.4
5.0
0.5
PVC (NeurIPS
22.5
35.7
3.8
16.2
7.0
MVCLN (CVPR'21)
35.7
16.2
13.9
13.9
34,2
10.9
17.0
15.7
4.4
24.3
12.4
24.3
10.0
5.7
23,0
20.8
9.5
80%
SURE (TPAMT23)
27.4
30.7
14.2
16.2
38.3
9.0
18.0
17.6
5.5
34,6
15.5
13.0
23.7
9.4
5.4
24.0
22.3
9.4
GCFAgg (CVPR'23) 23)
26.5
24.8
11.4
26.7
45.5
12.6
22
23.0
8.7
25.6
4.6
2.7
17.0
3.0
1.5
23.6
20.2
7.4
CGCN (TCSVT'24)
28.7
24.0
12.5
21.3
46.6
13.2
25.2
27.7
11.4
29.0
7.9
6.5
50.1
34.6
28.0
30.9
28.2
14.3
(AAAI'24)
27.8
27.1
12.
41.1
19.5
32.5
CANDY (Ours)
38.8
36.6
20.7
52.6
76.8
52.9
28.1
31.3
13.5
37.0
124
15.6
55.6
39.1
32.6
42.4
39.2
27.1
Table Clustering performance on incomplete multi-view datasets, in which 50% of samples are with missing views. The results are the mean of five individual runs. The best and second best results are shown bold and underline, respectively.
Scene15
Caltech101
Reuters
LandUse21
Average
Methods
ACC
ACC
ACC
ARI
ACC NMI
ARI
DCCAE (ICML' 15)
29.0
29.1
12.9
29.1
58.8
23.4
47.0
28.0
14.5
14.9
20.9
3.7
30.0
34.2
13.6
BMVC (TPAMI 18)
32.5
30.9
11.6
40.0
58.5
10.2
32
7.0
2.9
18.8
18.7
3.7
30.9
28.8
7.1
PMVC (AAAI'14 14)
25.5
25.4
50.3
74.5
41.5
29.3
4.4
20.0
23.6
8.0
3133
32.7
16.3
DAIMC (ICAA'18)
41.8
40.9
588
35.9
18.3
EERIMYC (TPAMI'2) 20)
28.9
27.0
8.4
43.6
69.0
26.4
29.8
12.0
42
22.1
25.2
9.1
31.1
33.3
12.0
SURE (TPAMI'22)
39.6
41.6
23.5
34.6
57.8
19.9
47.2
30.9
23.3
23.1
28.6
10.6
36.1
39.7
19.3
DSIMVC (ICML'22)
30.6
35.5
17.2
16.4
24.8
9.2
39.9
19.6
17.1
18.6
18.8
5.7
26.4
24.7
12.3
DCP (TPAMI'22)
39.5
42.4
23.5
44.3
71.0
45.3
34.6
17.5
2.9
22.2
27.0
10.4
35.2
39.5
20.5
Proimp (UCAI'23)
41.6
42.9
25.3
36.3
65.4
25.4
51.9
35.5
28.5
22.4
26.6
9.9
38.1
42.6
22.3
DIVIDE (AAAT'24)
46.8
45.7
29.1
63.4
82.5
$2.4
54.7
37.3
28.6
30.0
35.8
16.0
48.7
50.3
31.5
CANDY (Ours)
40.0
40.2
24.1
69.5
83.9
65.5
$4.2
34.8
27.2
28.8
31.1
14.4
48.1
47.5
32.8
8

---

Table Ablation studies on the Caltech- 101 and NUS- WIDE datasets with FP ratio of 20% and 50%. / represents using this component.
Caltech-101
NUS-WIDE
FP Ratio
Warmup Only
Re-alignment
SCD
CSM
ACC
NMI
ARI
ACC
NMI
ARI
46.9
67.5
29.5
57.5
37.9
33.1
49.9
70.9
32.3
58.6
39.8
34.7
56.5
78.4
38.4
58.1
43.6
37.0
65.0
82.3
60.1
60.3
47.1
36,9
54.2
22.5
44.2
17.2
30%
41.1
60.3
26.3
46.6
23.8
19.8
76.6
36.2
55.6
40.9
35.0
79.0
34.5
3400
Fostavo Potttes
Fodttre
Phatttre Palso Postave
2000
Poatave
3500
1200
3000
800
-000
1700
400
1000
0.6
1.5
(a) Warmup only
(b) Ours (50 epochs)
(c) Ours (Converged)
Figure 3: The normalized similarity distribution of true positive and false positive pairs.
Figures 5a and 5b demonstrate that our method robust to the selection of the denoising hyper- parameter 17. Notably, setting 7 too high would destroy the structural information of the high-order graph G (2)) Therefore, we fix 1) at 0.2 for all experiments without elaborated tuning.
4.4 Visualization on the Robustness (Working Mechanism)
To shed light the working mechanism behind CANDY, we visualize the achieved robustness against the false positive and negative correspondences, respectively. Fig. deepicts the distribution of true and false positive pairs, where one could observe that the SCD module could remarkably distinguish the noisy correspondence from the clean one, thus supporting the robustness against false positive correspondence. Mcanwhile, Fig. 4 presents the false negative recalling effects of different method variants, which demonstrate the significant semantic mining capacity of our CSM module and the polishing ability of the SCD module.
MSE=0.1023
ASSE=0.1394
MSE=0.0915
MSE=0.0890
MSE 0.0000
100
(a) Warmup Only
(b) DIVIDE 28
(c) Ours w/o SCD
Ours
(e) Ground Truth
Figure 4: The visualization of the cross-view similarity matrix, where each block is ordered using the ground-truth labels. For quantitative comparisons, we report the MSE between each result and the ground truth.
4.5 Comparisons with Noisy Correspondence Learning Approach (Approach Necessity)
As claimed in Related Works, we argue that the existing noisy correspondence learning cannot address the DNC problem well. In this section, verify the necessity to devise approach

---

Table 5: Performance comparisons between the SOTA noisy correspondence learning method (namely, RCL) and CANDY on handling the DNC problem. For a fair comparison, we adopt the same backbone (DIVIDE) for RCL as used CANDY.
dataset
Caltech-101
NUS-WIDE
method
DIVIDE
28
61
DIVIDE+OUIS
DIVIDE+RCL
DIVIDE+OurS
ratio
ACC
ARI
ACC
ARI
ACC
ARI
ACC
ARI
0.0
44.9
70.5
28.0
67.3
83,8
60.0
61.0
45.4
40.6
62.1
49.0
37.0
0.2
38.4
59.8
21.0
65,9
82.3
60.1
53.2
35.8
29.9
60.3
47.1
36.9
0.5
27.6
44,5
12.0
60.7
79.0
56.6
36.4
19.8
13.9
58.1
43.2
34.5
0.8
16.7
32.1
8.2
52.6
76.8
52.9
22.0
3.6
55.6
39.1
32.6
to the DNC problem. To this end, we adopt the SOTA noisy correspondence learning method 61 in the cross-modal retrieval area for the MvC task using the same architecture (namely, DIVIDE) as CANDY. Table summarizes the comparison results, highlighting the necessity of developing DNC-robust methods for contrastive MvC.
4.6 Study on the Generalizability (Approach Generalizability))
CANDY aims at generating a DNC-robust pseudo target for the existing contrastive MvC methods. To verify the generalizability of CANDY, in this section, we apply CANDY on another contrastive MvC baseline, namely, AECOKM. The results of "AECoKM" and "AECoKM+Ours" are shown in Fig. 5c where the two methods are conducted with the false positive ratio varying from 0.0 to 0.9 with an interval of 0.1. As one can observe, our CANDY could remarkably enhance the robustness and effectiveness of the baseline, demonstrating the plug-and-play role of our method.
ACC
70-
ACC
70-
60-
40
50-
30-
40-
10
0.1
02
033 0.4
055 06 Thrssbold
07
08
09
03 02
03
0.6
0.3
066
07 08 09
0.0 0.1
0.2
0.3 04
06 BP Ratio
0.5
0.7
0. 09
Thrsshold
(a) Caltech-101
(b) NEN-USWIIN WIDE
(c) CoAEKM NUS-WIDE
Figure 5: (a-b) Sensitivity studies of CANDY the hyper-parameter 7 for spectral denoising. (c) Investigation of the plug-and-pla role and robustness of CANDY, where AECoKM is another contrastive multi-view clustering (MvC) baseline which transferred CANDY.
Conclusion
In this paper, we reveal and study novel and practical problem within the field of contrastive Multi- view Clustering (MvC): Dual Noisy Correspondence (DNC). In brief, DNC involves both the false positive correspondences that arise during data collection, and the false negative correspondences that are inherent in the random sampling of contrastive MvC. To address this issue, we present CANDY comprising two novel modules: Context-based Semantic Mining (CSM) and Spectral- based Correspondence Denoising (SCD). On the one hand, CSM dexterously leverages contextual information transform distinct views into common contextual affinity space, thereby uncovering false negatives. On the other hand, SCD refines the pseudo target to mitigate the adverse impact of false positives by using the spectral denoising technique. By integrating these models, our method provides a auug-and-y most contrastive MyC methods against DNC. Extensive experiments on broad spectrum of scenarios have validated the effectiveness of CANDY. In the future, we plan to extend CANDY to address more practical scenarios, such as simultaneously handling both noisy correspondence and missing modalities.
the
solution that could enhance the robustness of
10

---

Acknowledgments
This work was supported in part by NSFC under Grant 62176171, U21B2040, 623B2075, and 62472295; in part by the Fundamental Research Funds for the Central Universities under Grant CJ202303 and CJ202403; and in part by Sichuan Science and Technology Planning Project under Grant 24NSFTD0130 and 2024NSFTD0047.
References
[1] Tadas Baltrušaitis, Chaitanya Ahuja. and Jouis-Philippe Morency. Multimodal machine learn- ing: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[2] Peng Xu, Xiatian Zhu, and David A. Clifton. Multimodal learning with transformers: A survey. IFFE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[3] Yiding Lu, Haobin Li, Yunfan Li, Yijie Lin, and Xi Peng. A survey on deep clustering: from the prior perspective. Vicinagearth, 2024.
[4] Chunlin Yu, Ye Shi, and Jingya Wang. Contextually affinitive neighborhood refinery for deep clustering. In Advances in Neural Information Processing Systems, 2023.
[5] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-viee representation learning. In Proceedings of the International Conference on Machine Learning, 2015.
[6] Yingming Li, Ming Yang, and Zhongfei Zhang. A survey of multi-view representation learning. IEEE Transactions on Knowledge and Data Engineering, 2018
[7] Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, and Jinxing Li. A survey on incomplete multiview clustering. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022.
[8] Shizhe Hu, Chengkun Zhang, Guoliang Zou, Zhengzheng Lou, and Yangdong Ye. Deep multiview clustering by pseudo-label guided contrastive learning and dual correlation learning. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[9] Erkun Yang. Mingxia Liu, Dongren Yao, Bing Cao, Chunfeng Lian, Pew-Thian Yap, and Dinggang Shen. Deep bayesian hashing with center prior for multi-modal neuroimage retrieval. IEEE Transactions on Medical Imaging, 2020.
[10] Xu Yang, Cheng Deng, Zhiyuan Dang, and Dacheng Tao. Deep multiview collaborative clustering. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[11] Hao Wang, Yan Yang, and Bing Liu. GMC: traph-based multi-view clustering. IEEE Transac- tions on Knowledge and Data Engineering, 2020.
[12] Daniel Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering representation alignment for multi-view clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[13] Daniel J Trosten, Sigurd Løkse, Robert Jenssen, and Michael C Kampffmeyer. On the effects of self-supervision and contrastive alignment in deep multi-view clustering. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
[14] Erlin Pan and Zhao Kang. Multi-view contrastive graph clustering. In Advances in Conference on Neural Information Processing Systems, 2021.
[15] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun- Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vasion-language representation learning with noisy text supervision. In Proceedings of the International Conference on Machine Learning, 2021.
11

---

[16] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, 2021.
[17] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2018.
[18] Mouxing Yang. Yunfan Li, Peng Hu, Jinfeng Bai, Jiancheng Lx, and Xi Peng. Robust multi-view clustering with incomplete information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[19] Antoine Miech, Dimitri Zhukov, Jean-Baatisee Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivie. HowTo100M: Learning a Text- Video Embedding by Watching Hundred Million Narrated Video Clips. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019.
[20] Zhenyu Huang, Peng Hu, Joey Tianyi Zhou, Jiancheng Lx, and Xi Peng. Partially view-aligned clustering. In Advances in Neural Information Processing Systems, 2020.
[21] Mouxing Yang, Yunfan Li, Zhenyu Huang, Zitao Liu, Peng Hu, and Xi Peng. Partially view-aligned representation learning with noise-robust contrastive loss. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[22] Ajit Rajwade, Anand Rangarajan, and Arunava Banerjee. Image denoising using the higher order singular value decomposition. IEEE Transactions Pattern Analysis and Machine Intelligence, 2013.
[23] Qiang Guo, Caiming Zhang, Yunfeng Zhang, and Hui Liu. An efficient svd- baddd method for image denoising. IEEE Transactions on Circuits and Systems for Video Technology, 2016.
[24] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Proceedings of the European Conference on Computer Vision, 2020.
[25] Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. COMPLETER: Incomplete multi-view clustering via contrastive prediction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
[26] Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. Partial multi-vien clustering via consistent gan. In Proceedings of the IEEE International Conference Data Mining, 2018
[27] Xingfeng Li, Yinghui Sun, Quansen Sun, Zhenwen Ren, and Yuan Sun. Cross-view graph matching guided anchor alignment for incomplete multi-view clustering. Information Fusion, 2023.
[28] Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, and Xi Peng. Decoupled contrastive multi-view clustering with high-order random walks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
[29] Xinran Ma, Mouxing Yang, Yunfan Li, Peng Hu, Jiancheng Lv, and Xi Peng. Cross-modal retrieval with noisy correspondence via consistency refining and mining. IEEE Transactions on Image Processing, 2024.
[30] Zhenyu Huang, Guocheng Niu, Xiao Liu, Wenbiao Ding, Xinyan Xiao, Hua Wu, and Xi Peng. Learning with noisy correspondence for cross-modal matching. In Advances in Neural Informa- tion Processing Systems, 2021.
[31] Yang Qin, Dezhong Peng. Xi Peng. Xu Wang, and Peng Hu. Deep evidential learning with noisy correspondence for cross-modal retrieval. Proceedings of the ACM International Conference on Multimedia, 2022.
12

---

[32] Mouxing Yang, Zhenyu Huang, and Xi Peng. Robust object re- re-identification with coupled noisy labels. International Journal of Computer Vision 2024.
[33] Yang Qin, Yingke Chen, Dezhong Peng, Xi Peng, Joey Tianyi Zhou, and Peng Hu. Noisy- correspondence learning for text-to-image person re-identification. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2024.
[34] Mouxing Yang. Zhenyu Huang, Peng Hu, Taihao Li, Jiancheng Lv, and Xi Peng. Learning with twin noisy labels for visible-infrared person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[35] Yuan Sun, Yang Qin, Yongxiang Li, Dezhong Peng, Xi Peng, and Peng Hu. Robust multi-view clustering with noisy correspondence. IEEE Transactions Knowledge and Data Engineering. 2024.
[36] Yijie Lin, Mouxing Yang, Jun Yu, Peng Hu, Changqing Zhang, and Xi Peng. Graph matching with bi-level noisy correspondence. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
[37] Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, Zujie Wen, and Xi Peng. Multi-granularig correspondence learning from long-term noisy videos. In Proceedings of the International Conference on Learning Representations, 2024.
[38] Zhenyu Huang, Mouxing Yang. Xinyan Xiao, Peng Hu, and Xi Peng. Noise-robust vision- language pre-training with positive-negative learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[39] Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning, 2020.
[40] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[41] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, 2015.
[42] Li Fei-Fei and Pietro Perona. A bayesian hierarchical model for learning natural scene categories. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
[43] Yeqing Li, Feiping Nie, Heng Huang, and Junzhou Huang. Large-scale multi-view spectral clustering via bipartite graph. In Proceedings of the AAAI Conference Artificial Intelligence, 2015.
[44] Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view classification. In Proceedings of the International Conference Learning Representations, 2020.
[45] Yi Yang and Shawn Newsam. Bag-of-visual-worre and spatial extensions for land-use classifica- tion. In Proceedings of the SIGSPATIAL International Conference Advances in Geographic Information Systems, 2010.
[46] Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[47] Massih R. Amini, Nicolas Usunier, and Cyril Goutte. Learning from multiple partially observed views -an application to multilingual text categorization. In Proceedings of the International Conference on Neural Information Processing Systems, 2009.
[48] Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, and Jiancheng Lvv. Multi-view spectral clustering network. In Proceedings of the International Joint Conference on Artificial Intelligence, 2019.
13

---

[49] Peng Hu, Liangli Zhen, Dezhong Peng, and Pei Liu. Scalable deep multimodal learning for cross-modal retrieval. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Reirieval, 2019.
[50] Liangl Zhen, Peng Hu, Xu Wang, and Dezhong Peng. Deep supervised cross-modal retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[51] Weiran Wang. Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view representation learning. In Proceedings of the International Conference on Machine Learning, 2015.
[52] Zheng Zhang, Li Liu, Fumin Shen, Heng Tao Shen, and Ling Shao. Binary multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[53] Yiming Wang, Dongxia Chang, Zhiqiang Fu, Jie Wen, and Yao Zhao. Partially view-aligned representation learning via cross-view graph contrastive network. IEEE Transactions on Circuits and Systems for Video Technology, 2024.
[54] Weiqing Yan, Yuanyang Zhang, Chenlei Lv, Chang Tang abd Guanghui Yue, Liang Liao, and Weisi Lin. GCFAgg: Global and cross-view feature aggregation for multi-view clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
[55] Shao- Yuan Li, Yuan Jiang, and Zhi-Hua Zhou. Partial multi-view clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, 2014.
[56] Menglei Hu and Songean Chen. Doubly aligned incomplete multi-view clustering. In Proceed- ings of the International Joint Conference on Artificial Intelligence, 2018.
[57] Xinwang Liu, Miaomiao Li, Chang Tang, Jingyuan Xia, Jian Xiong, Li Liu, Marius Kloft, and En Zhu, Efficient and effective regularized incomplete multi-view clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[58] Huayi Tang and Yong Liu. Deep safe incomplete multi-view clustering: Theorem and algorithm. In Proceedings of the International Conference on Machine Learning, 2022.
[59] Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
[60] Haobin Li, Yunfan Li, Mouxing Yang, Peng Hu, Dezhong Peng, and Xi Peng. Incomplete mieti-view clustering via prototype-based imputation. In Proceedings of the International Joint Conference Artificial Intelligence, 2023.
[61] Peng Hu, Zhenyu Huang, Dezhong Peng, Xu Wang, and Xi Peng. Cross-modal retrieval with partially mismatched pairs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The scope and contributions of this paper are clearly stated in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

This paper is experimental instead of theoretical. It includes no theorems or proofs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: This paper provides a detailed algorithm, instructions on dataset and prepro- cessing procedures, hardware and software configurations for the reproducibility of the proposed method.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: This study uses public datasets. The URL of the code repository is included in the Abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
mublic//uuiees//ddeetbbmsss wotpssiiinoollyyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wttps/cccppplcccuui.eeaeeecccuea
Stteee//CooSSSemsssS.Si.a wsw.sssoiccgyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper provide the details of data preprocessing steps, hyperparameters and optimization in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Almost all compared baselines do not include the statistical significance in experiments thus we do not report it.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computer resources is stated in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: fttttp::///eurrie pwwrnpsu...../. hctpppplllc..tcccccceee......aaa
Answer: [Yes]
Justification:
This paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The proposed algorithm have no societal impact. All datasets used in this paper are publicly available, and the algorithm only produces clustering assignments.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: All datasets used in this paper are publicly available, and the algorithm only produces clustering assignments.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Original papers and datasets are properly cited.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
21


Reviews:
[{"review_id": "GAgdTIWylG", "reviewer": "Reviewer_jTED", "summary": "The manuscript \"Robust Contrastive Multi-view Clustering against Dual Noisy Correspondence\" addresses the Dual Noisy Correspondence (DNC) issue in contrastive multi-view clustering (MvC), where noise affects both positive and negative data pairs. The authors propose CANDY (Contextually-spectral based correspondence refinery), a method that uses inter-view similarities to identify false negatives and a spectral-based module to denoise correspondences, thus mitigating the impact of false positives. Extensive experiments on five multi-view benchmarks demonstrate CANDY's effectiveness over existing methods.", "questions": "How is the pseudo target C defined in Equation 1 of the proposed method? Please provide a detailed explanation of its construction and role within the CANDY framework.", "limitations": "Please see weaknesses.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "-The paper introduces the DNC problem, a novel and practical challenge in MvC, highlighting the dual nature of noise affecting both positive and negative pairs.\n\n-CANDY combines two innovative components: the Context-based Semantic Mining (CSM) module, which identifies false negatives using high-order contextual affinities, and the Spectral-based Correspondence Denoising (SCD) module, inspired by signal processing techniques, to filter out false positives. This dual approach ensures robust clustering by comprehensively addressing both types of noise.\n\n-The proposed method consistently outperforms state-of-the-art multi-view clustering methods on various benchmarks, demonstrating its robustness and effectiveness. The extensive experiments, varying the ratio of false positives, provide thorough validation of CANDY’s capabilities.", "weaknesses": "-The method's applicability to other types of multi-view learning tasks, such as classification or retrieval, is not explored, limiting its broader impact within the multi-view learning domain. Including classification results would enhance the paper.\n\n-Visualization results, such as those in Figure 3, should be compared with other baseline methods to provide detailed insights into how CANDY's clustering performance or robustness visually differs from that of other methods, particularly in handling noisy correspondences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive reviews and suggestions. We will address your questions one by one.\n\n> ***Question 1**: The method's **applicability to other types of multi-view learning tasks**, such as classification or retrieval, is not explored, limiting its broader impact within the multi-view learning domain. Including classification results would enhance the paper.*\n\nThank you for your constructive suggestion. We follow [A] to endow our method with the capacity to handle the classification problem. To achieve multi-view classification, we randomly select 80% of the data as the training set, and use the remaining 20% as the test set. Then we simulate 50% noisy correspondence in the training set, while keeping the test set clean. After training the model using our CANDY method as the feature extractor, we train a support vector machine (SVM) on the features of the train set. Finally, we use the SVM to predict the labels on the features of the test set. The results are summarized in the following table.\n\n| Model | Caltech-101 | NUS-WIDE  |\n| - | - | - |\n| SURE [A] | 56.65 | 48.68 |\n| DIVIDE [B] | **76.12** | 50.06  |\n| CANDY (Ours) | 72.97 | **61.02** |\n\nOur method achieves competitive results on these datasets, justifying the applicability of our method to the classification task.\n\n[A] Robust Multi-View Clustering With Incomplete Information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.\n\n[B] Decoupled Contrastive Multi-view Clustering with High-order Random Walks. AAAI Conference on Artificial Intelligence, 2024.\n\n> ***Question 2**: **Visualization results, such as those in Figure 3, should be compared with other baseline methods** to provide detailed insights into how CANDY's clustering performance or robustness visually differs from that of other methods, particularly in handling noisy correspondences.*\n\nAs per your suggestions, we have supplemented the visualization comparisons in Figures S2 and S3 of the rebuttal PDF file, comparing our CANDY with the most competitive baseline DIVIDE [B]. As observed from Figure S2, DIVIDE struggles to differentiate between true and false positives during training, indicating overfitting on noisy correspondence. In contrast, Figure S3 demonstrates that our CANDY achieves a clear separation between true and false positives, highlighting its superior robustness in handling noisy correspondences.\n\n> ***Question 3**: **How is the pseudo target C defined in Equation 1** of the proposed method? Please provide a detailed explanation of its construction and role within the CANDY framework.*\n\nWe apologize for the unclear description regarding the construction of the pseudo target $\\mathbf{C}$. $\\mathbf{C}$ serves as the target of the contrastive loss, which is defined as\n$$\n\\mathcal{L} = \\sum_{v_1=1}^V \\sum_{\\substack{v_2=1 \\\\ v_2 \\neq v_1}}^V \\mathcal{H}\\left(\\mathbf{C}^{(v_1,v_2)}, \\rho\\left(\\mathbf{Z}^{(v_1)}{\\mathbf{Z}^{(v_2)}}^\\top \\right)\\right),\n$$\nwhere $\\mathcal{H}$ denotes the row-wise cross-entropy function with mean reduction, $\\rho\\left(\\cdot \\right)$ denotes the softmax function, $\\mathbf{C}^{(v_1,v_2)} \\in \\mathbb{R}^{n \\times n}$ and $\\mathbf{Z}^{(v_1)}{\\mathbf{Z}^{(v_2)}}^\\top$ represent the pseudo targets and affinity matrix between views $v_1$ and $v_2$, respectively. In general, traditional contrastive multi-view clustering methods assume that the cross-view correspondence is faultless, typically adopting an identity matrix $\\mathbf{I}\\in \\mathbb{R}^{n \\times n}$ as the target. As verified in our experiments, such a vanilla target not only misleads the model to overfit false positives but also neglects numerous semantically associated false negatives. Therefore, we propose the following formulation of $\\mathbf{C}$ to achieve robustness against such a dual noisy correspondence problem:\n$$\n\\mathbf{C} = \\widetilde{\\mathbf{G}} + \\varepsilon \\mathbf{I},\n$$\nwhere $\\varepsilon$ is a hyper-parameter set to 0.2 for all the experiments, and $\\widetilde{\\mathbf{G}}$ is the denoised pseudo target obtained through our context-based semantic mining and spectral-based correspondence denoising modules as elaborated in the manuscript. For your convenience, we attach the key steps for constructing the denoised target $\\widetilde{\\mathbf{G}}$ below. The pseudo target $\\widetilde{\\mathbf{G}}$ is obtained by\n$$\n\\widetilde{\\mathbf{G}}^{(v_1 \\rightarrow v_2)} = \\mathbf{U} \\mathbf{\\widetilde{\\Sigma}} \\mathbf{V^{\\top}},\n$$\n\nwhere $\\widetilde{\\mathbf{\\Sigma}} = \\mathrm{diag}(\\lambda_1, \\ldots, \\lambda_L, 0, \\ldots, 0) \\in \\mathbb{R}^{n \\times n}$ is a diagonal matrix consisting of the retained singular values $( \\lambda_1 > \\cdots > \\lambda_L \\ge \\eta)$, with a denoising hyper-parameter $\\eta$ set to $0.2$ in our experiments. To obtain $\\widetilde{\\mathbf{\\Sigma}}$, $\\mathbf{U}$ and $\\mathbf{V}$, we decompose $\\mathbf{G}^{(v_1 \\rightarrow v_2)}$ by Singular Value Decomposition (SVD) as\n\n$$\n\\mathbf{G}^{(v_1 \\rightarrow v_2)} = \\mathbf{U} \\mathbf{\\Sigma} \\mathbf{V^{\\top}},\n$$\nwhere $\\mathbf{\\Sigma} = \\mathrm{diag}(\\lambda_1, \\ldots, \\lambda_n)$ denotes a diagonal matrix consisting of the singular values, and columns of $\\mathbf{U}$ and $\\mathbf{V}$ are the left- and right-singular vectors, respectively. The cross-view high-order affinity graph $\\mathbf{G}^{(v_1 \\rightarrow v_2)}$ is defined as\n$$\n\\mathbf{G}^{(v_1 \\rightarrow v_2)} = \\mathbf{A}^{(v_1 \\rightarrow v_2)} {\\mathbf{A}^{(v_2 \\rightarrow v_2)}}^{\\top},\n$$\n\nwhere the affinity graph $\\mathbf{A}^{(v_1 \\rightarrow v_2)}$ from $v_1$ to $v_2$ is constructed by samples $\\mathbf{Z}^{(v_1)}$ and $\\mathbf{Z}^{(v_2)}$ as nodes in a minibatch, with edge weights defined by Gaussian kernel similarity. Specifically, we formulate the connection probability from one node to all others as a context, which serves as an embedding of the node $i$ for semantic mining, with\n$$\n\\mathbf{A}_{i j}^{(v_1 \\rightarrow v_2)}=\\exp (-\\|[\\mathbf{Z}^{(v_1)}]_i-[\\mathbf{Z}^{(v_2)}]_j\\|^2 / \\sigma).\n$$", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V9O9tNlqAx", "reviewer": "Reviewer_L9C1", "summary": "This paper addresses a new problem called dual noisy correspondence, which the authors claim is practical and underexplored in the multi-view learning community. Dual noisy correspondence refers to two challenges: 1) false positive correspondences induced by irrelevant multi-view data and 2) false negative correspondences caused by the random sampling characteristic of contrastive learning. To address this problem, the authors propose a novel metric for cross-view similarity estimation that recalls more false negative pairs, and a spectral-based denoising method to address false positive correspondences. Extensive experiments on multiple datasets validate the superiority of the proposed method compared to existing approaches. Overall, this paper makes an important contribution by tackling the practical and underexplored problem of dual noisy correspondence in multi-view learning. The technical solutions proposed, including the new similarity metric and spectral denoising, demonstrate strong empirical performance.", "questions": "(1) What is the novelty of the proposed setting?\n\n (2) What is the additional time cost?", "limitations": "see weakness", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper introduces a novel perspective on cross-view similarity measurement. Specifically, the authors observe that the row-wise similarity in the first-order cross-view similarity matrix, referred to as context in the paper, can serve as an effective metric for similarity estimation. Experiments show that this approach recalls more potential false negative pairs.\nExtensive experiments on five benchmarks with four types of noise ratios comprehensively verify the effectiveness of the proposed method.", "weaknesses": "The authors claim to address a new problem, dual noisy correspondence (false positive and false negative). However, reviewers noted that similar issues have been explored in works [A] of other fields. Additional claims and discussions are needed to clarify the novelty of the proposed setting, which the authors regard as a significant contribution. [A] Cross-modal retrieval with noisy correspondence via consistency refining and mining, TIP.\n-How about the time cost of the proposed method? It is essential to compare this with other methods, particularly the peer method (e.g., DIVIDE) with the same network architecture.\nThe reviewer suggests that the noise ratios for both false positive and false negative correspondences should be explicitly presented in the main table for a more comprehensive understanding of performance and comparisons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable review. We will address your questions one by one.\n\n> _**Question 1**: The authors claim to address a new problem, dual noisy correspondence (false positive and false negative). However, reviewers noted that similar issues have been explored in works [A] of other fields. Additional claims and discussions are needed to **clarify the novelty of the proposed setting**, which the authors regard as a significant contribution._\n\nThank you for pointing out the related work [A], which addresses both the false positive and false negative correspondence for the image-text retrieval task. Although [A] shares some similarities with our work, the definitions of false positives and false negatives are remarkably different. First, [A] is specialized for the retrieval task where the false positives emerge i.f.f. the given two samples do not belong to the same instance. In contrast, in our work, false positives are defined as wrongly matched pairs consisting of samples from different classes. Second, [A] seeks to mine semantic-consistent samples and regards them as false negatives, whereas our work considers arbitrary cross-view within-class samples as false negatives. Intuitively, given a false positive/negative pair, our work has a probability of $\\frac{1}{K}$ for both false positive re-alignment and false negative recalling, while the probabilities for [A] are $\\frac{1}{N}$, where $K$ is the number of classes and $N$ is the number of instances in the dataset. Thanks to the differences, it is difficult or even impossible to realign mismatched pairs in [A], while our method can achieve this for a much smaller $K$.\n\n[A] Cross-modal Retrieval with Noisy Correspondence via Consistency Refining and Mining. IEEE Transactions on Image Processing, 2024.\n\n> ***Question 2**: How about the **time cost** of the proposed method? It is essential to compare this with other methods, particularly the peer method (e.g., DIVIDE) with the same network architecture.*\n\nAs you suggested, we run DIVIDE and our CANDY with the same network architecture for a fair comparison, and report their running time (in seconds, per epoch) in the following table.\n\n| Model        | Caltech101 | LandUse21 | NUSWIDE | Reuters | Scene15 |\n| ------------ | ---------- | --------- | ------- | ------- | ------- |\n| DIVIDE       | 0.1438     | 0.0647    | 0.2286  | 0.4617  | 0.1346  |\n| CANDY (Ours) | 2.4971     | 0.7010    | 2.1663  | 4.6905  | 1.2743  |\n\nAs indicated in the table, CANDY takes approximately 10 times the computational time per epoch compared to DIVIDE. The additional computational overhead is mainly attributed to the Singular Value Decomposition (SVD) procedure in our approach. In the future, we plan to explore lightweight and efficient alternatives for SVD.\n\n> ***Question 3**: The reviewer suggests that the **noise ratios for both false positive and false negative correspondences should be explicitly presented in the main table** for a more comprehensive understanding of performance and comparisons.*\n\nAs per your suggestions, we have given the factual noise ratios for both false positives (FPs) and false negatives (FNs) in the following table. Specifically, we set one view as the anchor view, and simulate FP samples by randomly shuffling the other view by a given percentage, denoted by FP ratio $\\eta$. FN is inherent to the dataset. For example, the FN ratio of a dataset is $1 / K$ if its samples are evenly distributed in $K$ classes.\n\n| $\\eta$ | Caltech101 |      | LandUse21 |      | NUSWIDE |      | Reuters |       | Scene15 |      |\n| ------ | ---------- | ---- | --------- | ---- | ------- | ---- | ------- | ----- | ------- | ---- |\n|        | FP         | FN   | FP        | FN   | FP      | FN   | FP      | FN    | FP      | FN   |\n| 0.0    | 0.00       | 2.84 | 0.00      | 4.73 | 0.00    | 9.99 | 0.00    | 21.40 | 0.00    | 6.91 |\n| 0.2    | 19.34      | 2.84 | 19.10     | 4.73 | 17.98   | 9.99 | 15.67   | 21.40 | 18.68   | 6.91 |\n| 0.5    | 48.45      | 2.84 | 47.33     | 4.73 | 45.07   | 9.99 | 39.64   | 21.40 | 46.42   | 6.91 |\n| 0.8    | 77.48      | 2.84 | 76.24     | 4.73 | 72.03   | 9.99 | 62.57   | 21.40 | 73.98   | 6.91 |\n\nIn the next version, we will detail the simulation procedure of noisy correspondence and the practical noise ratios in our experiments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o03RWGdTXN", "reviewer": "Reviewer_6oUm", "summary": "The authors delve into the study of contrastive multi-view clustering (MVC) and aim to address the false positive and false negative correspondence issues, collectively referred to as dual noisy correspondence. To tackle this problem, they propose a two-fold solution named CANDY. Firstly, CANDY exploits inter-view similarities as context to uncover false negatives. Secondly, it employs a spectral-based module to denoise correspondences, thereby mitigating the negative impact of false positives.\n\nOne of the most interesting insights in this paper is the observation that context can serve as a new axis for transforming data similarity into a high-order affinity space. \n\nTo verify the effectiveness of CANDY, the authors conduct experiments across various datasets and settings.", "questions": "Please see the weaknesses.", "limitations": "No.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "i) This paper is well-written and structured, making it accessible to readers even outside the immediate research community. Additionally, the experiment designs are intriguing, and the extensive experiments convincingly demonstrate the effectiveness of the proposed methods and the two corresponding modules.\n\nii) The proposed method is technically sound. Moreover, it can serve as a plug-and-play module that can be integrated into other contrastive MVC methods, enhancing their robustness.", "weaknesses": "i) I have some doubts regarding the results in Figure 4. Firstly, how is the cross-view similarity matrix arranged? The current form appears to be arranged by the ground truth, whereas my understanding is that the ground truth should be agnostic during training. Secondly, how is robustness compared in the figure? While the proposed method can recall more false negative samples, it seems to introduce more wrongly recalled samples. How is the trade-off between truly and falsely recalled samples balanced?\n\nii)  Regarding the references in Line 31, is the observation in Fig. 1(a) first proposed in this paper?\n\niii) Two open-world questions. Firstly, robustness against incomplete views is another important topic in the MVC community. Some baselines in the paper address this issue. Can the proposed method be extended to handle incomplete views? If so, this would significantly enhance the paper’s contribution. Secondly, the approach is mainly presented in scenarios involving two views. How could the method be extended to handle more views (≥ 3)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive reviews and suggestions. Below, we will address each of your questions.\n\n> ***Question 1.1**: I have some doubts regarding the results in Figure 4. Firstly, **how is the cross-view similarity matrix arranged**? The current form appears to be arranged by the ground truth, whereas my understanding is that the ground truth should be agnostic during training.*\n\nThe cross-view similarity matrix depicted in Figure 4 is arranged according to the ground-truth labels. To be specific, we first compute the similarities of cross-view representations and then arrange them according to the labels. In other words, each diagonal block in the matrix represents the similarities of within-class samples. Thanks to the matrix re-arrangement manner, it allows us to investigate the false negatives recall capacity by comparing the consistency between the cross-view similarity matrix and the ground-truth one. Notably, the labels are only utilized for the above analysis experiment, while keeping agnostic during both the training and inference phases.\n\n> ***Question 1.2**: Secondly, **how is robustness compared in the figure**? While the proposed method can recall more false negative samples, it seems to introduce more wrongly recalled samples. How is the trade-off between truly and falsely recalled samples balanced?*\n\nTo quantitatively compare the robustness between the baselines and our method, we additionally report the Mean Squared Error (MSE) to measure the difference between each cross-view similarity matrix and the ground-truth one in the figure. More specifically, the smaller MSE value indicates the greater robustness of the method against false negatives. Notably, as truly and falsely recalled samples in local regions cannot reflect the global difference, it is sub-optimal to measure the robustness by directly observing the cross-view similarity matrix. In contrast, the MSE value seeks a trade-off between truly and falsely recalled samples from the global perspective, thus being a promising metric to demonstrate the robustness against false negatives.\n\n> ***Question 2**: Regarding the references in Line 31, **is the observation in Fig. 1(a) first proposed in this paper**?*\n\nYes, our work could be the first study to reveal the dual noisy correspondence (DNC) challenge in the multi-view clustering community. In brief, DNC refers to the noise present in both cross-view positive and negative pairs. Although the references in Line 31 reveal the noise in the training multimodal data, they either do not explicitly handle the noise [A,B,D,E], or only conquer false negative issue [C].\n\n[A] Learning transferable visual models from natural language supervision. International Conference on Machine Learning, 2021.\n\n[B] Scaling up visual and vision-language representation learning with noisy text supervision. International Conference on Machine Learning, 2021.\n\n[C] Robust multi-view clustering with incomplete information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.\n\n[D] Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. Proceedings of ACL, 2018.\n\n[E] HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips. International Conference on Computer Vision, 2019.\n\n> ***Question 3.1**: Firstly, robustness against incomplete views is another important topic in the MVC community. Some baselines in the paper address this issue. Can the proposed method be extended to **handle incomplete views**? If so, this would significantly enhance the paper's contribution.*\n\nThank you for your constructive suggestions. As you suggested, we follow DIVIDE [F] to endow our method with the capacity of handling incomplete views. In brief, we utilize the cross-view decoders to recover the missing samples and perform k-means on all the latent representations to achieve incomplete multi-view clustering. The results are summarized in the following table, where the results of baseline methods are copied from the DIVIDE paper.\n\n| Incomplete (50% missing) | Model| Scene15||| Caltech101 ||| Reuters||| LandUse21 ||| Average|||\n| -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |\n||| ACC| NMI| ARI| ACC| NMI| ARI| ACC| NMI| ARI| ACC | NMI| ARI| ACC| NMI| ARI|\n|| SURE | 39.6 | 41.6 | 23.5 | 34.6 | 57.8 | 19.9 | 47.2 | 30.9 | 23.3 | 23.1| 28.6 | 10.6 | 36.1 | 39.7 | 19.3 |\n| | DIVIDE | **46.8** | **45.7** | **29.1** | 63.4 | 82.5 | 52.4 | **54.7** | **37.3** | **28.6** | **30.0**| **35.8** | **16.0** | **48.7** | **50.3** | 31.5 |\n|| CANDY (Ours) | 40.0 | 40.2 | 24.1 | **69.5** | **83.9** | **65.5** | 54.2 | 34.8 | 27.2 | 28.8| 31.1 | 14.4 | 48.1 | 47.5 | **32.8** |\n\nFrom the results, one could observe that our method achieves promising performance in handling the incomplete view problem, even though it is primarily designed to address the dual noisy correspondence challenge.\n\n[F] Decoupled Contrastive Multi-view Clustering with High-order Random Walks. AAAI Conference on Artificial Intelligence, 2024.\n\n> ***Question 3.2**: Secondly, the approach is mainly presented in scenarios involving two views. **How could the method be extended to handle more views (≥ 3)**?*\n\nAccording to your comment, we extend CANDY to scenarios involving more than two views. Specifically, following [G], we conduct experiments on the Caltech7 dataset using two views (Caltech7-2V) and three views (Caltech7-3V). The results, summarized in the following table, demonstrate CANDY's capability in handling multiple views. \n\n| Method       | FP ratio | Caltech7-2V | Caltech7-3V |\n| ------------ | -------- | ----------- | ----------- |\n| CANDY (Ours) | 0.0      | 52.16       | 60.37       |\n|              | 0.5      | 41.96       | 47.93       |\n\n[G] Multi-level Feature Learning for Contrastive Multi-view Clustering. The IEEE / CVF Computer Vision and Pattern Recognition Conference, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YZ18TQkagf", "reviewer": "Reviewer_G6Ay", "summary": "The paper presents a novel method, Contextually-spectral based correspondence refinery (CANDY), to address the Dual Noisy Correspondence (DNC) problem in contrastive multi-view clustering (MvC). CANDY utilizes inter-view similarities as context and employs a spectral-based module for denoising correspondence, effectively mitigating the influence of false positives and uncovering false negatives. The method's effectiveness is demonstrated through extensive experiments on five multi-view benchmarks, outperforming eight competitive MvC methods.", "questions": "Please refer to weaknesses.", "limitations": "Please refer to weaknesses.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- CANDY introduces a new perspective to handle DNC, which is practical and challenging in MvC. This paper demonstrated potential capacity as a plug-and-play solution to enhance the robustness of existing contrastive MvC methods.\n- The combination of context-based semantic mining and spectral-based denoising provides a robust solution against noisy correspondence. Extensive experiments and comparisons with state-of-the-art methods validate the effectiveness of CANDY.", "weaknesses": "1. How will the choice of the Gaussian kernel parameter $\\sigma$ influence the performance of CANDY? The sensitivity of the clustering results to this parameter needs to be thoroughly examined, as different values of $\\sigma $ might affect the affinity graph’s construction.\n2. Can the authors provide insights into the computational overhead introduced by the CANDY method? \n3. How are the false negative and false positive proportions handled in the experiments? Further clarification is needed on how this proportion is maintained across different datasets and experimental conditions. Does the false negative proportion keep fixed as 1/K in Table 1?\n4. An interesting question arises from the paper’s claim that CANDY can serve as a plug-and-play solution. Is it possible to integrate the pseudo target generated by CANDY into other contrastive multi-view clustering methods beyond DIVIDE? Exploring this potential would demonstrate the versatility of CANDY and its applicability to a broader range of clustering techniques, providing valuable insights into how the pseudo target can enhance other methods’ robustness against noisy correspondences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your acknowledgment of our method. We will address your questions one by one.\n\n> ***Question 1**: **How will the choice of the Gaussian kernel parameter σ influence the performance** of CANDY? The sensitivity of the clustering results to this parameter needs to be thoroughly examined, as different values of σ might affect the affinity graph's construction.*\n\nThank you for your suggestions. As you suggested, we conduct parameter analysis experiments to investigate the influence of $\\sigma$ on the Caltech-101 dataset by varying the value of $\\sigma$ from [0.01, 0.23] with an interval of 0.02. The results are summarized in the following table.\n\n| Dataset  \\\\ $\\sigma$ | 0.01  | 0.03  | 0.05      | 0.07  | 0.09  | 0.11  | 0.13      | 0.15  | 0.17  | 0.19  | 0.21  | 0.23  |\n| -------------------- | ----- | ----- | --------- | ----- | ----- | ----- | --------- | ----- | ----- | ----- | ----- | ----- |\n| Caltech-101          | 16.13 | 16.75 | **60.97** | 60.71 | 60.31 | 51.41 | 47.22     | 41.97 | 37.14 | 34.72 | 32.37 | 32.98 |\n\nFrom the results, one could observe that our method performs stably in the range of [0.05, 0.09] for $\\sigma$. Thus, we simply set the Gaussian kernel parameter $\\sigma$ as 0.07 for all experiments in the paper. For a clear observation, we include the plot of the result as Figure S1 in the rebuttal PDF.\n\n> ***Question 2**: Can the authors provide insights into the **computational overhead** introduced by the CANDY method?*\n\nAs you suggested, we run DIVIDE and our CANDY with the same network architecture for a fair comparison, and report their running time (in seconds, per epoch) in the following table.\n\n| Model        | Caltech101 | LandUse21 | NUSWIDE | Reuters | Scene15 |\n| ------------ | ---------- | --------- | ------- | ------- | ------- |\n| DIVIDE       | 0.1438     | 0.0647    | 0.2286  | 0.4617  | 0.1346  |\n| CANDY (Ours) | 2.4971     | 0.7010    | 2.1663  | 4.6905  | 1.2743  |\n\nAs indicated in the table, CANDY takes approximately 10 times the computational time per epoch compared to DIVIDE. The additional computational overhead is mainly attributed to the Singular Value Decomposition (SVD) procedure in our approach. In the future, we plan to explore lightweight and efficient alternatives for SVD.\n\n> ***Question 3:** How are the false negative and false positive proportions handled in the experiments? Further clarification is needed on **how this proportion is maintained across different datasets and experimental conditions**. Does the false negative proportion keep fixed as 1/K in Table 1?*\n\nAs per your suggestions, we have given the factual noise ratios for both false positives (FPs) and false negatives (FNs) in the following table. Specifically, we set one view as the anchor view, and simulate FP samples by randomly shuffling the other view by a given percentage, denoted by FP ratio $\\eta$. FN is inherent to the dataset. For example, the FN ratio of a dataset is $1 / K$ if its samples are evenly distributed in $K$ classes.\n\n| $\\eta$ | Caltech101 |      | LandUse21 |      | NUSWIDE |      | Reuters |       | Scene15 |      |\n| ------ | ---------- | ---- | --------- | ---- | ------- | ---- | ------- | ----- | ------- | ---- |\n|        | FP         | FN   | FP        | FN   | FP      | FN   | FP      | FN    | FP      | FN   |\n| 0.0    | 0.00       | 2.84 | 0.00      | 4.73 | 0.00    | 9.99 | 0.00    | 21.40 | 0.00    | 6.91 |\n| 0.2    | 19.34      | 2.84 | 19.10     | 4.73 | 17.98   | 9.99 | 15.67   | 21.40 | 18.68   | 6.91 |\n| 0.5    | 48.45      | 2.84 | 47.33     | 4.73 | 45.07   | 9.99 | 39.64   | 21.40 | 46.42   | 6.91 |\n| 0.8    | 77.48      | 2.84 | 76.24     | 4.73 | 72.03   | 9.99 | 62.57   | 21.40 | 73.98   | 6.91 |\n\nIn the next version, we will detail the simulation procedure of noisy correspondence and the practical noise ratios in our experiments.\n\n> ***Question 4**: An interesting question arises from the paper's claim that CANDY can serve as a plug-and-play solution. Is it possible to **integrate the pseudo target generated by CANDY into other contrastive multi-view clustering methods** beyond DIVIDE?*\n\nOur research has considered the broader applicability of CANDY, and we have conducted experiments to demonstrate its integration with other methods beyond DIVIDE. In Section 4.6 of our manuscript, we present the results of integrating CANDY with another model, AECoKM, on the NUS-WIDE dataset. Figure 5(c) summarizes the performance improvements observed with AECoKM under different false positive (FP) ratios. For your convenience, we have converted the figure as a table and attached it below.\n\n| FP Ratio      |     | 0.0   | 0.1   | 0.2   | 0.3   | 0.4   | 0.5   | 0.6   | 0.7   | 0.8   | 0.9   |\n| ------------- | --- | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- | ----- |\n| AECoKM        | ACC | 15.80 | 15.70 | 15.86 | 16.94 | 18.36 | 16.86 | 17.64 | 18.02 | 17.27 | 18.37 |\n|               | NMI | 2.56  | 2.56  | 2.38  | 3.67  | 5.62  | 3.36  | 4.28  | 4.27  | 3.59  | 4.16  |\n|               | ARI | 1.12  | 1.14  | 1.10  | 1.57  | 2.42  | 1.49  | 1.98  | 2.15  | 1.84  | 2.20  |\n| AECoKM + Ours | ACC | 57.88 | 50.25 | 55.68 | 56.72 | 54.89 | 54.82 | 53.85 | 56.12 | 54.93 | 53.44 |\n|               | NMI | 46.83 | 39.04 | 45.27 | 44.27 | 41.94 | 42.48 | 40.98 | 42.61 | 41.43 | 40.51 |\n|               | ARI | 37.98 | 30.63 | 36.03 | 36.21 | 33.94 | 34.37 | 33.49 | 35.22 | 34.14 | 33.00 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear ACs and Reviewers,

We sincerely appreciate your time and effort in reviewing our paper and providing constructive feedback. We thank the reviewers for your recognition of our novelty and contributions.

* This method serves as a plug-and-play module that can be integrated into other contrastive MVC methods [6oUm, G6Ay]
* The proposed method consistently outperforms state-of-the-art multi-view clustering methods on various benchmarks [jTED, L9C1, G6Ay].
* This paper is well-written and structured [6oUm].

In the following sections, we have addressed each concern and query raised by the reviewers during the rebuttal phase. We have also included a **PDF document with additional figures** to complement our responses.

Thank you again for your valuable feedback and time investment.

Best regards,

The Authors

Meta-review: This paper makes a valuable contribution to the multi-view clustering (MvC) community by introducing the Dual Noisy Correspondence (DNC) problem and proposing a novel solution based on a contextually-spectral mechanism. The work is supported by strong experimental validation and demonstrates significant technical innovation. During the rebuttal process, the authors effectively addressed the reviewers' concerns, leading two out of four reviewers to raise their scores to clear acceptance. Overall, there is no dispute regarding the paper's contribution and technical novelty, and I recommend its acceptance.
Decision: Accept (poster)