Title: Evaluate then Cooperate: Shapley-based View Cooperation Enhancement for Multi-view Clustering
Venue: NeurIPS 2024
Accepted: yes
Keywords: Multi-view Clustering

Clustering Network
Abstract: The fundamental goal of deep multi-view clustering is to achieve preferable task performance through inter-view cooperation. Although numerous DMVC approaches have been proposed, the collaboration role of individual views have not been well investigated in existing literature. Moreover, how to further enhance view cooperation for better fusion still needs to be explored. In this paper, we firstly consider DMVC as an unsupervised cooperative game where each view can be regarded as a participant. Then, we introduce the Shapley value and propose a novel MVC framework termed Shapley-based Cooperation Enhancing Multi-view Clustering (SCE-MVC), which evaluates view cooperation with game theory. Specially,  we employ the optimal transport distance between fused cluster distributions and single view component as the utility function for computing shapley values. Afterwards, we apply shapley values to assess the contribution of each view and utilize these contributions to promote view cooperation. Comprehensive experimental results well support the effectiveness of our framework adopting to existing DMVC frameworks, demonstrating the importance and necessity of enhancing the cooperation among views.

Evaluate then Cooperate: Shapley-based View Cooperation Enhancement for Multi-view Clustering

Abstract

The fundamental goal of deep multi-view clustering is to achieve preferable task performance through inter-view cooperation. Although numerous DMVC approaches have been proposed, the collaboration role of individual views have not been well investigated in existing literature. Moreover, how to further enhance view cooperation for better fusion still needs to be explored. In this paper, we firstly consider DMVC as an unsupervised cooperative game where each view can be regarded as a participant. Then, we introduce the Shapley value and propose a novel MVC framework termed S hapley-based C ooperation E nhancing M ultiv iew C lustering ( SCE-MVC ), which evaluates view cooperation with game theory. Specially, we employ the optimal transport distance between fused cluster distributions and single view component as the utility function for computing shapley values. Afterwards, we apply shapley values to assess the contribution of each view and utilize these contributions to promote view cooperation. Comprehensive experimental results well support the effectiveness of our framework adopting to existing DMVC frameworks, demonstrating the importance and necessity of enhancing the cooperation among views.

1 Introduction

Recently, multi-view clustering has become one of the most prominent problems in unsupervised learning[1, 2, 3, 4]. It leverages data from different views, combining them to provide richer information for clustering tasks. Generally speaking, multi-view clustering can be broadly categorized into traditional methods [5, 6, 7, 8] and deep multi-view clustering (DMVC) [5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] methods. Compared to the limited feature extraction capabilities of traditional methods, deep multi-view clustering methods better preserve information from different views through flexible deep neural networks. For most DMVC methods, the key challenge is how to enhance cooperation among views to accomplish better clustering performance[20].

Numerous DMVC algorithms have already achieved significant results in downstream clustering tasks. However, there has been little research on investigating the the contribution of each view in the fusion stage. Once evaluating the view contribution, we may observe the phenomenon: One view dominates the fusion process, suppressing the collaborative contribution of other views, leading to suboptimal clustering results. We attribute this phenomenon to insufficient cooperation between views, and thus propose a fundamental assumption: More balanced contributions and more extensive cooperation among multiple views can lead to better clustering results, as shown in Fig. 1.

/          View 1,2 Contribution with SCE

/          View 1,2 Contribution without SCE

/          Fusion ACC with / without SCE

0.75

0.55

0.

45

0.35

0.25

0.15

0.65

Epochs

/       /        View 1,2,3 Contribution with SCE

/       /        View 1,2,3 Contribution without SCE

/           Fusion ACC with / without SCE

0.

94

0.

92

0.

90

0.88

0.

86

0.

84

0.

82

0.

80

Epochs

(a)

/       /        View 1,2,3 Contribution with SCE

/       /        View 1,2,3 Contribution without SCE

/           Fusion ACC with / without SCE

0.

94

0.

92

0.

90

0.88

0.

86

0.

84

0.

82

0.

80

/          View 1,2 Contribution with SCE

/          View 1,2 Contribution without SCE

/          Fusion ACC with / without SCE

0.75

0.55

0.

45

0.35

0.25

0.

15

0.65

Epochs

Epochs

(b)

Caltech101-7dataset

UCI-digit dataset

1.0

0.6

0.8

View Contribution

View Contribution

0.5

FusionACC

0.6

0.4

0.4

0.3

0.2

0.2

0.0

0

20

40

60

80

100

0

100

200

300

400

500

600

To this end, we consider introducing the concept of game theory into multi-view clustering and propose the S hapley-based C ooperation E nhancing (SCE) method. Shapley value[21, 22] is a concept used to measure the contribution of participants in game theory[23, 24]. It utilizes a utility function to evaluate the different marginal contributions that each participant brings when combined with other participants, and calculates their contribution to the overall cooperation by taking the weighted average over all possible combinations of participants[21]. Upon quantifying the contribution of each view, the cooperation among views can be analyzed based on their respective contributions. By employing a module that enhances view cooperation, we can dynamically adjust the parameter convergence rate of views during training in accordance with their contributions. This enables the underrepresented views to receive a more balanced training regimen, thereby enhancing post-fusion performance[25].

The primary contributions of this work can be summarized as follows:

Our work is the first framework to evaluate the view contributions within an unsupervised context. Leveraging the proposed View Contribution Evaluation Module, we theoretically quantify the individual contributions of views.

We propose the View Cooperation Enhancing Module based on the view contributions obtained by the View Contribution Evaluation Module, allowing suppressed views to effectively participate in the fusion process, ultimately enhancing clustering performance.

Comprehensive experiments have been conducted to showcase the versatility of our SCE framework, which can effectively evaluate the contributions of views across various MVC frameworks.

2 Related Work

2.1 Deep Multi-view Clustering

In deep multi-view clustering[26, 27, 28, 29], deep neural networks[30, 31] with multiple nonlinear transformations are more effective at acquiring feature representations than traditional shallow models. Presently, deep multi-view clustering methods fall into three main categories[32, 33]: joint methods[34, 35, 36], alignment-based methods[14, 37, 38, 39], and other methods[40, 41, 42]. Joint methods consider differences and complementarities among views, optimizing sample representations within each view's space. For example, the DMJC[35] utilizes autoencoders to generate view representations, refining them as pseudo-labels for self-supervised optimization. Alignment-based methods focus on consistency between views, mapping representations into a shared subspace. These methods often employ contrastive learning, such as ProImp[43], which integrates contrastive learning

at both sample and prototype levels. Some approaches combine joint and alignment-based methods, like the share-specific method[40], which separates sample representations into shared and specific components, aligns shared representations into a common space, and refines specific representations in separate spaces.

2.2 Rethinking of Cooperation in DMVC

The core concept of DMVC is to enhance clustering performance by fostering enhanced cooperation and utilizing complementary information across views. Although existing DMVC research emphasizes leveraging inter-view cooperation for improved clustering tasks, there is currently a lack of research quantifying the contribution of views during the fusion stage to enhance cooperation. Multi views[20] and multi modalities[44] are two closely intertwined domains. Within the realm of multimodal fusion, studies have surfaced highlighting the issue of subpar collaboration among modalities, prompting endeavors to alleviate the imbalance in modality cooperation. Wang et al. [45] uncovered varying convergence rates among diverse modalities, leading to a scenario where jointly trained multimodal models struggle to match or outperform their unimodal counterparts. Furthermore, Peng et al. [25] demonstrated that in the context of audio-visual learning, superior performance in the audio modality hampers the optimization of the video modality.

Drawing inspiration from prior studies, our research aims to evaluate the individual contributions of each view in multi-view clustering fusion. Nevertheless, evaluating view contribution based solely on labels in unsupervised environments is infeasible. Moreover, employing weights as a measure for view contribution may not align with all DMVC structures, especially those based on contrastive learning frameworks. Consequently, quantifying view contributions in fusion and fostering their cooperation pose significant research challenges.

2.3 Shapley Value: a Method for Evaluating Contribution.

In order to reasonably and effectively evaluate the contributions of each view in the fusion process, we refer to the theoretical framework of game theory. The Shapley value [46] within game theory proves instrumental in quantifying participants' contributions to cooperative coalitions. Denote X = { x i } n i =1 as the alliances of n participants. For a participant x i , the Shapley value considers x i 's contribution in every set that includes him. Let S i = { S ⊆ X| x i ∈ S } represent the set of all subsets of X that include the participant x i , then, the overall contribution of member x i , i.e. , the Shapley value, can be expressed as

where | s | signifies the cardinality of set s ; v ( · ) represents the utility function; [ υ ( s ) -υ ( s \{ i } )] quantifies the marginal contribution of x i to set s ; and ( | s |-1)!( n -| s | )! n ! denotes the weight assigned to this marginal contribution, determined by the probability of set s occurring.

Given the advantageous characteristics of Shapley values-efficiency, symmetry, dummy, and additivity-researchers have increasingly employed them for explaining machine learning models. For instance, Lundberg et al. [47] proposed a method to interpret predictions by computing feature importance based on Shapley values. Hu et al. [48] applied Shapley values to evaluate modality contributions in supervised tasks, while Wei et al. [49] explored modality assessment using Shapley values. However, these studies focus mainly on supervised scenarios, making the definition of utility functions for Shapley values in unsupervised settings a challenging task yet to be resolved.

3 Method

3.1 Problem Statement

Given a multi-view dataset X = { X (1) , X (2) , ..., X ( V ) } , where V is the number of views, and X ( v ) ( v = 1 , 2 , ..., V ) denotes the original feature space of the v -th view. Consider the clustering problem on the dataset, each cluster is represented by a clustering center µ ( v ) j , j = 1 , ..., K , where K is the number of clusters. To achieve better clustering performance, we map the original feature space X ( v ) to a latent embedded feature space Z ( v ) through a non-linear mapping function

BaselineMvCModel

①Shapley-basedViewContributionEvaluationModule

View 1

View V

q°←→p

↑P

*8'0

0.8

Shapley-based

Encoder

Encoder

0.4=

0.2

View Contribution

of View 1

Fusion

Decoder

Decoder

0.8

Shapley-based

View Contribution

DataDistribution

of ViewV

s0

0.6=

BEFORE

②Shapley-basedView

Coordination

AFTER

View1 View2  View3 View4

View1 Contribution

Enhancing Module

View 1 Contribution

w/ SCE

w/o SCE

View 2 Contribution

View 2 Contribution

w/o SCE

w/ SCE

strengthe

[k.ve(v)

w/o SCE

View N Contribution

View N Contribution

w/ SCE

0.2

0.4

0.6

0.8

View Contribution

f ( v ) θ ( υ ) : X ( v ) → Z ( v ) , where θ ( v ) represents the learnable parameters of mapping function for the v -th view.

3.2 View Contribution Evaluation Module

Modified Shapley Value for Multi Views

To assess the contributions of each view in MVC tasks, we treat each view as a player and introduce the Shapley value referring to Eq. (1). Let X = { X ( i ) } V i =1 be the set of all views, S v = {S ⊆ X| X ( v ) ∈ S} represent the set of all subsets of X that include the v -th view X ( v ) . Define an internal metric of clustering E independently of labels to serve as a utility function, which will be elaborated on in the next subsection. Then the Shapley value of the v -th view can be calculated as

where | s | denotes the number of views in the view-set s .

Utility Function in Shapley Value

In Shapley value, the utility function is used to evaluate the benefit created by a coalition. In DMVC, where each view acts as a participant, the utility function needs to be designed to reflect the contribution of any combination of views in fusion, without the need for label assistance. In this scenario, we constructed a novel function E using the following method.

Inspired by the DEC model [50], we assume that the data from each view follows a Student's tdistribution. Following the approach outlined by [51], we utilize the t-distribution as a kernel function to evaluate the distance between the sample embeddings z i and the cluster centers µ j within a view:

where α represents the degrees of freedom in the t -distribution. q ij denotes the probability of assigning the feature z i to the cluster center µ j .

Fix the degrees of freedom α to 1, the cluster distribution of multi-view data q can be considered as a combination of single-view cluster distribution based on a weight matrix π :

where the weight matrix π ∈ R K × V represents the importance of the cluster centers µ j in different views. This weight matrix π can be obtained by performing row normalization on an unconstrained trainable matrix W :

For any given combination of an m -view union U = { X ( u 1 ) , X ( u 2 ) , ..., X ( u m ) } , we can obtain the weight matrix π U by selecting the corresponding columns in the matrix W . By applying Eq. (3), we derive the data distribution for the fused view, which is denoted as q U .

In particular, if U = ∅ , which means no view is involved in the clustering process, we consider q ∅ to be a uniform distribution. This means that the probability of any embedding z i belonging to any cluster µ j is equal and given by 1 /K , where K is the total cluster numbers. In this case, we denote the resulting data distribution q ∅ as

where 1 N × K ∈ R N × K is a matrix in which all elements are equal to 1.

According to the principle of self-distillation in DEC, we sharpen the fused distribution q X from all views to obtain a more powerful target distribution p :

Since we consider the distribution p as the trend of the fusion process, the closer the fusion distribution q U (combined with U = { X ( u 1 ) , X ( u 2 ) , ..., X ( u m ) } ) is to the distribution p , the higher the participation of view set U in the fusion process. Therefore, we can use the optimal transport distance from distribution q U to distribution p as a measure of confidence for distribution q U . Additionally, based on our desire for the metric to possess monotonicity and boundedness, we define the utility function based on distribution transportation as:

where OT ( · , · ) is calculated as wasserstein distance and E ( · ) ∈ (0 , 1] . The closer the data distribution is to the pseudo-labels p , the larger the value of the metric E ( · ) .

In the process of computing the optimal transport OT ( · , · ) , we consider the distribution q as a probability set where K cluster centers are assigned by N samples. The probability of each cluster center is 1 /K . For example, when calculating OT ( q { X (1) } , q { X (2) } ) , the distribution of views can be represented as [1 /K, 1 /K,..., 1 /K ] T , and the distance matrix { D ij } K × K can be calculated as

where the j -th cluster center for the v -th view can be represented as ξ ( v ) j = { q ( v ) ij } N × 1 .

Shapley-based View Contribution: a Formal Representation

In the above subsections, we have already discussed the integration of cooperative game theory with multi-view clustering tasks. We have also defined a utility function E that reflects the accuracy of clustering. Now, combining the Eq. (2) and (8), we present the formal representation of viewcontribution through Shapley values:

The sum of all view contributions is then calculated as:

As p and q X evolve throughout training, the sum of Shapley values for all views also fluctuate accordingly. To obtain the contribution proportions of each view to the fused view, let

where φ v is the normalized Shapley value, determining the respective contribution of different views.

3.3 Theoretical Analysis

To further demonstrate the generalization performance of the View Contribution Evaluation Module, we apply it to the alignment-based method and the joint method, respectively. The relevant theoretical derivations are given below.

Theorem 1: For alignment-based methods, considering the representation { z (1) i } N i =1 and { z (2) i } N i =1 on two views, with the infoNCE contrastive learning loss:

where N is the sample number and τ > 0 is a scalar temperature hyperparameter. ℓ con (1 , 2) brings the representations between views closer, leads to the average of view contributions, i.e. , | ϕ 1 -ϕ 2 |→ 0 .

Theorem 2: For joint methods, considering the representation Z (1) = f (1) θ (1) ( X (1) ; θ (1) ) and Z (2) = f (2) θ (2) ( X (2) ; θ (2) ) on two views, with a simple view-level weight fusion during the fusion process: Z = w 1 Z (1) + w 2 Z (2) . There are situations where view 2 is dominated by view 1 and cannot make its contribution, i.e. , w 1 ≫ w 2 .

Remark: Theorem 1 indicates that alignment-based methods tend to align the contributions of different views. Under our assumption, the cooperation between views is already sufficient, optimizing the model based on contributions can only lead to limited improvements. Theorem 2 suggests that in joint methods, one view may suppress another, highlighting insufficient cooperation between views. In such cases, enhancing cooperation between views theoretically improves model performance. We will elaborate on the View Cooperation Enhancing Module in Section 3.4 and validate these conclusions through experiments. The proofs for Theorems 1 and 2 can be found in Appendix.

3.4 View Cooperation Enhancing Module

After acquiring contributions of different views, we seek to enhance their cooperation by dynamically regulating views' training speeds according to these contributions. This proportional control guarantees active involvement of all views in the integration process. For this purpose, we introduce the convergence speed ratio, labeled as k , derived from the contributions of each view.

To ensure the applicability of our S hapley-based C ooperation E nhancing( SCE ) method to most multi-view clustering tasks, we consider a typical multi-view clustering framework: firstly, obtain embeddings for each view by utilizing autoencoders; then, fuse the embeddings from different views using a global optimization goal. Let f ( v ) θ ( v ) ( X ( v ) ; θ ( v ) ) represent the encoder model for the v -th view, g denote the fusion method for multiple views, and l represent the loss function for computing the fusion loss. The overall fusion loss L can be represented as

In practice, the parameters for the v -th view is updated as

With the normalized Shapley values calculated by Eq. (12) to dynamically monitor the contribution differences between different views, we are able to adaptively modulate the gradients via an empirical formula:

where τ is a temperature hyperparameter to control the degree of modulation. Thus k v is not greater than 1 , and the larger the contribution of the view, the smaller the k v . We integrate the coefficient k i into the Adam optimization method, and the update of θ ( v ) t +1 at iteration t is as follows:

where k v is used as a relative convergence speed ratio for the v -th view in gradient descent.

Theorem 3: For joint methods that use view-level weight fusion Z = w 1 Z (1) + w 2 Z (2) , where Z (1) = f (1) θ (1) ( X (1) ; θ (1) ) and Z (2) = f (2) θ (2) ( X (2) ; θ (2) ) , gradient modulation in Eq. (18) allows the two views to contribute more evenly, i.e. , w 1 w 2 → 1 .

Remark: Theorem 3 indicates that through dynamic gradient adjustments, we can harmonize the learning progress among different views, attain a more average distribution of contributions, and enhance the cooperation among views. The proof for Theorem 3 can be found in Appendix.

Algorithm 1: S hapley-based C ooperation E nhancing M ulti-view C lustering( SCE-MVC )

Input:

Multi-view dataset { X } , temperature hyperparameter τ

( v ) V v =1 ˆ

Output:

// Warm-up training

Obtain initial cluster assignments through a selected algorithm;

// Optimization with SCE module while the fusion loss has not converge do

// View Contribution Evaluation

Obtain shapley value for each view with Eq. (4), (7) and (10);

Normalize the shapley value as view contribution with Eq. (12);

// View Cooperation Enhancing

Calculate the convergence speed ratio k v for each view with Eq. (17);

Update the parameters of each view relatively with Eq. (18);

return;

3.5 Implementation

This subsection outlines the implementation of our SCE module, detailed in Algorithm 1. After obtaining initial cluster center assignments, we iteratively employ the following two modules:

View Contribution Evaluation . First, we calculate the cluster distribution for each individual view as well as the combined view. Next, we compute an enhanced distribution p . After utilizing the optimal transport distance between distributions as the utility function, we can calculate view contributions.

View Cooperation Enhancing . We utilize the calculated contribution values to influence the training process of each view using a convergence ratio k . The larger the contribution value of a view, the less the convergence ratio k will be. This will coordinate the training process of the views, enabling better cooperation among views, and allowing them to collaborate more effectively.

4 Experiments

In this section, we implement experiments on alignment-based method and joint method to verify the effectiveness of the proposed theorems and SCE module by addressing the following questions:

RQ1 : For alignment-based MVC methods, do the contributions of views calculated by the View Contribution Evaluation Module exhibit relative uniformity, thereby validating Theorem 1 ?

RQ2 : In the case of uniform view contributions, can model performance be further enhanced by the View Cooperation Enhancing Module?

RQ3 : For joint methods, are extreme view contributions present, thereby validating Theorem 2 ?

RQ4 : Can model performance be enhanced by harmonizing view contributions through View Cooperation Enhancing Module, thereby validating Theorem 3 ?

RQ5 : How do the hyper-parameter τ impact the performance of SCE-MVC?

4.1 Datasets, Metrics and Experimental settings

Our experiments utilized six multi-view datasets, including CUB 2 , Caltech101-7 3 , HandWritten 4 , UCI-digit 5 , STL10 6 and Reuters 7 . The detailed information of these datasets is listed in the Table 1. Meanwhile, three widely used metrics are adopted in our experiment, including clustering accuracy (ACC), normalized mutual information (NMI) and adjusted rand index(ARI).

For fairness, We conduct all experiments on PyTorch platform using the NVIDIA 2060 GPU. Besides, ten state-of-the-art MVC methods are introduced: DEMVC[13], CoMVC[14], SiMVC[14], SDSNE[15], MFLVC[16], SDMVC[17], DSMVC[18], APADC[52], DMJC[35] and ProImp[43].

4.2 Evaluate on Alignment-based Methods(RQ1 & RQ2)

In this section, we apply the SCE methods to two alignment-based MVC methods:

Utilizing infoNCE as a contrastive loss and employing Kmeans after concatenation.

Incorporating contrastive learning at both the sample and prototype levels as ProIMP[43].

We applied these two approaches to the CUB and Caltech101-7 datasets, each comprising two views. Detailed experimental results are presented in the Table 2. The results lead to following conclusions:

Caltech101

UCI-digit

STL10

w/o SCE

w/o SCE

w/o SCE

w/ SCE

w/ SCE

w/ SCE

View1

View2

View3

View4

View Contribution

0

0.2

1

0.8

0.6

0.4

0.25

0.35

0.45

0.55

0.65

0.75

0.85

0.95

Caltech101

UCI-digit

STL10

w/o SCE

w SCE

ACC

(a)

Caltech101

UCI-digit

STL10

w/o SCE

w/o SCE

w/o SCE

w/ SCE

w/ SCE

w/ SCE

View1

View2

View3

View4

View Contribution

0

0.2

1

0.8

0.6

0.4

0.25

0.35

0.45

0.55

0.65

0.75

0.85

0.95

Caltech101

UCI-digit

STL10

w/o SCE

w SCE

ACC

(b)

Assessed with the View Contribution Evaluation Module, the contributions of the two views in alignment-based methods are remarkably similar, validating the assertion of Theorem 1.

As the contributions of the views tend to equalize, the utilization of the View Cooperation Enhancing Module approach only yields subtle improvements to the alignment-based methods.

4.3 Evaluate then Cooperate on Joint Methods(RQ3 & RQ4)

In this subsection, we apply the SCE model to a joint MVC methods(DMJC[35]). We have carried out comprehensive experiments on six datasets with varying numbers of views, as depicted in Table 3 and Fig. 3. In Table 3, we showcase the performance of SCE-MVC in comparison to ten benchmark methods. Fig. 3 illustrates the ablation study of DMJC with and without SCE. Based on the aforementioned experimental results, we draw the following conclusions:

Fig. 3(a) illustrates the changes in view contributions with and without the SCE module across three datasets. Prior to the use of the SCE module, there is a scenario where certain views dominate while others are suppressed, a pattern particularly evident in the Caltech101-7 dataset, aligning well with the conclusions of Theorem 2.

After introducing the View Contribution Enhancing Module, the contributions of the views become more balanced, leading to a significant improvement in the model's performance. Specifically, the ACC, NMI, and ARI metrics on the Caltech101-7 dataset all witness an increase of over 10 percentage points, thus validating the conclusion of Theorem 3. More detailed contribution comparison data can be found in the Appendix.

4.4 Sensitive Analysis(RQ5)

In SCE-MVC, the singular hyperparameter τ is essential for fine-tuning the convergence equilibrium among distinct views according to their contributions. In sensitive analysis, we varied τ within { 0 . 5 , 1 , 1 . 5 , 2 , 2 . 5 , 3 } , as illustrated in Table 4. As τ increases, the changes in clustering results metrics become less pronounced, which can be attributed to the characteristics of the exponential functions in Eq. (17). Specifically, as τ increases, the convergence ratio k v tends to be closer to zero.

5 Conclusion

This paper explores the roles of views in multi-view clustering through the lens of game theory. We introduce a novel contribution evaluation module founded on the Shapley value. Our experiments showcase the adaptability of this module across mainstream MVC frameworks, accurately assessing each view's impact. By scrutinizing the functions of diverse views during fusion, we devise a cooperation enhancing module to bolster cooperation among views. Extensive dataset validations underscore the effectiveness and robust applicability of our approach and theorems.

Acknowledgments and Disclosure of Funding

This work is supported by National Natural Science Foundation of China under Grant No. 62276271, 62325604, 62406329, 62476280, 62476281 and National Science and Technology Innovation 2030 Major Project under Grant No. 2022ZD0209103.

References

Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. Multi-view subspace clustering. In Proceedings of the IEEE international conference on computer vision , pages 4238-4246, 2015.

Xinwang Liu, Xinzhong Zhu, Miaomiao Li, Lei Wang, Chang Tang, Jianping Yin, Dinggang Shen, Huaimin Wang, and Wen Gao. Late fusion incomplete multi-view clustering. IEEE transactions on pattern analysis and machine intelligence , 41(10):2410-2423, 2018.

Siwei Wang, Xinwang Liu, Xinzhong Zhu, Pei Zhang, Yi Zhang, Feng Gao, and En Zhu. Fast parameter-free multi-view subspace clustering with consensus anchor guidance. IEEE Transactions on Image Processing , 31:556-568, 2021.

Xinwang Liu. Simplemkkm: Simple multiple kernel k-means. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):5174-5186, 2022.

Wei Xia, Quanxue Gao, Qianqian Wang, Xinbo Gao, Chris Ding, and Dacheng Tao. Tensorized bipartite graph learning for multi-view clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Wenjue He, Zheng Zhang, Yongyong Chen, and Jie Wen. Structured anchor-inferred graph learning for universal incomplete multi-view clustering. World Wide Web , 26(1):375-399, 2023.

Shudong Huang, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. Measuring diversity in graph learning: a unified framework for structured multi-view clustering. IEEE Transactions on Knowledge and Data Engineering , 34(12):5869-5883, 2021.

Qian Qu, Xinhang Wan, Weixuan Liang, Jiyuan Liu, Yu Feng, Huiying Xu, Xinwang Liu, and En Zhu. An lightweight anchor-based incremental framework to multi-view clustering. In ACM Multimedia 2024 .

Erlin Pan and Zhao Kang. Multi-view contrastive graph clustering. Advances in neural information processing systems , 34:2148-2159, 2021.

Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. Comic: Multiview clustering without parameter selection. In International conference on machine learning , pages 5092-5101. PMLR, 2019.

Junpu Zhang, Liang Li, Siwei Wang, Jiyuan Liu, Yue Liu, Xinwang Liu, and En Zhu. Multiple kernel clustering with dual noise minimization. In Proceedings of the 30th ACM International Conference on Multimedia , MM'22, page 3440-3450, New York, NY, USA, 2022. Association for Computing Machinery.

Daniel J Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering representation alignment for multi-view clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1255-1265, 2021.

Jie Xu, Yazhou Ren, Guofeng Li, Lili Pan, Ce Zhu, and Zenglin Xu. Deep embedded multi-view clustering with collaborative training. Information Sciences , 573:279-290, 2021.

Daniel J Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering representation alignment for multi-view clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1255-1265, 2021.

Chenghua Liu, Zhuolin Liao, Yixuan Ma, and Kun Zhan. Stationary diffusion state neural estimation for multiview clustering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7542-7549, 2022.

Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. Multi-level feature learning for contrastive multi-view clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16051-16060, 2022.

Jie Xu, Yazhou Ren, Huayi Tang, Zhimeng Yang, Lili Pan, Yang Yang, Xiaorong Pu, S Yu Philip, and Lifang He. Self-supervised discriminative feature learning for deep multi-view clustering. IEEE Transactions on Knowledge and Data Engineering , 2022.

Huayi Tang and Yong Liu. Deep safe multi-view clustering: Reducing the risk of clustering performance degradation caused by view increase. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 202-211, 2022.

Fangdi Wang, Siwei Wang, Jiaqi Jin, Zhibin Dong, Xihong Yang, Yu Feng, Xinzhong Zhu, Tianrui Liu, Xinwang Liu, and En Zhu. View gap matters: Cross-view topology and information decoupling for multi-view clustering. In ACM Multimedia 2024 .

Uno Fang, Man Li, Jianxin Li, Longxiang Gao, Tao Jia, and Yanchun Zhang. A comprehensive survey on multi-view clustering. IEEE Transactions on Knowledge and Data Engineering , 2023.

Eyal Winter. The shapley value. Handbook of game theory with economic applications , 3:2025-2054, 2002.

Luke Merrick and Ankur Taly. The explanation game: Explaining machine learning models using shapley values. In Machine Learning and Knowledge Extraction: 4th IFIP TC 5, TC 12, WG8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2020, Dublin, Ireland, August 25-28, 2020, Proceedings 4 , pages 17-38. Springer, 2020.

John Von Neumann and Oskar Morgenstern. Theory of games and economic behavior, 2nd rev. 1947.

Drew Fudenberg and Jean Tirole. Game theory . MIT press, 1991.

Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8238-8247, 2022.

Daniel J Trosten, Sigurd L o kse, Robert Jenssen, and Michael C Kampffmeyer. On the effects of self-supervision and contrastive alignment in deep multi-view clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23976-23985, 2023.

Shiye Wang, Changsheng Li, Yanming Li, Ye Yuan, and Guoren Wang. Self-supervised information bottleneck for deep multi-view subspace clustering. IEEE Transactions on Image Processing , 32:1555-1567, 2023.

Huayi Tang and Yong Liu. Deep safe multi-view clustering: Reducing the risk of clustering performance degradation caused by view increase. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 202-211, 2022.

Zhenyu Huang, Peng Hu, Joey Tianyi Zhou, Jiancheng Lv, and Xi Peng. Partially view-aligned clustering. Advances in Neural Information Processing Systems , 33:2892-2902, 2020.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems , 25, 2012.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Xiaodong Jia, Xiao-Yuan Jing, Qixing Sun, Songcan Chen, Bo Du, and David Zhang. Human collective intelligence inspired multi-view representation learning-enabling view communication by simulating human communication mechanism. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Yingming Li, Ming Yang, and Zhongfei Zhang. A survey of multi-view representation learning. IEEE transactions on knowledge and data engineering , 31(10):1863-1883, 2018.

Wei Xia, Sen Wang, Ming Yang, Quanxue Gao, Jungong Han, and Xinbo Gao. Multi-view graph embedding clustering network: Joint self-supervision and block diagonal representation. Neural Networks , 145:1-9, 2022.

Yuan Xie, Bingqian Lin, Yanyun Qu, Cuihua Li, Wensheng Zhang, Lizhuang Ma, Yonggang Wen, and Dacheng Tao. Joint deep multi-view learning for image clustering. IEEE Transactions on Knowledge and Data Engineering , 33(11):3594-3606, 2020.

Lusi Li, Zhiqiang Wan, and Haibo He. Incomplete multi-view clustering with joint partition and graph learning. IEEE Transactions on Knowledge and Data Engineering , 2021.

Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning , pages 4116-4126. PMLR, 2020.

Junnan Li, Pan Zhou, Caiming Xiong, and Steven CH Hoi. Prototypical contrastive learning of unsupervised representations. arXiv preprint arXiv:2005.04966 , 2020.

Huasong Zhong, Chong Chen, Zhongming Jin, and Xian-Sheng Hua. Deep robust clustering by contrastive learning. arXiv preprint arXiv:2008.03030 , 2020.

Jie Xu, Yazhou Ren, Huayi Tang, Xiaorong Pu, Xiaofeng Zhu, Ming Zeng, and Lifang He. Multi-vae: Learning disentangled view-common and view-peculiar visual representations for multi-view clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9234-9243, 2021.

Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. Consistent and specific multiview subspace clustering. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018.

Le Ou-Yang, Xiao-Fei Zhang, Dao-Qing Dai, Meng-Yun Wu, Yuan Zhu, Zhiyong Liu, and Hong Yan. Protein complex detection based on partially shared multi-view clustering. BMC bioinformatics , 17:1-15, 2016.

Haobin Li, Yunfan Li, Mouxing Yang, Peng Hu, Dezhong Peng, and Xi Peng. Incomplete multi-view clustering via prototype-based imputation. arXiv preprint arXiv:2301.11045 , 2023.

Tadas Baltru v saitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence , 41(2):423-443, 2018.

Weiyao Wang, Du Tran, and Matt Feiszli. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12695-12705, 2020.

Lloyd S Shapley et al. A value for n-person games. 1953.

Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems , 30, 2017.

Pengbo Hu, Xingyu Li, and Yi Zhou. Shape: An unified approach to evaluate the contribution and cooperation of individual modalities. arXiv preprint arXiv:2205.00302 , 2022.

Yake Wei, Ruoxuan Feng, Zihe Wang, and Di Hu. Enhancing multi-modal cooperation via fine-grained modality valuation. arXiv preprint arXiv:2309.06255 , 2023.

Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International conference on machine learning , pages 478-487. PMLR, 2016.

Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(11), 2008.

Jie Xu, Chao Li, Liang Peng, Yazhou Ren, Xiaoshuang Shi, Heng Tao Shen, and Xiaofeng Zhu. Adaptive feature projection with distribution alignment for deep incomplete multi-view clustering. IEEE Transactions on Image Processing , 32:1354-1366, 2023.

Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning , pages 9929-9939. PMLR, 2020.

Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert RG Lanckriet, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In Proceedings of the 18th ACM international conference on Multimedia , pages 251-260, 2010.

Changqing Zhang, Huazhu Fu, Qinghua Hu, Xiaochun Cao, Yuan Xie, Dacheng Tao, and Dong Xu. Generalized latent multi-view subspace clustering. IEEE transactions on pattern analysis and machine intelligence , 42(1):86-99, 2018.

A Proof of Theorems

A.1 Proof of Theorem 1

Theorem 1: For alignment-based methods, considering the representation { z (1) i } N i =1 and { z (2) i } N i =1 on two views, with the infoNCE contrastive learning loss:

where N is the sample number and τ > 0 is a scalar temperature hyperparameter. ℓ con (1 , 2) brings the representations between views closer, leads to the average of view contributions, i.e. , | ϕ 1 -ϕ 2 |→ 0 .

Proof: Referring to Definition of Perfect Alignment in [53], we say the two views are perfectly aligned if z (1) i = z (2) i , i = 1 , 2 , ..., N . Assuming that with the training of contrastive learning, the representations of the two views tend to align perfectly. Then, there exists a limiting representation Z ∗ such that lim ∥ Z 1 -Z ∗ ∥ = lim ∥ Z 2 -Z ∗ ∥ = 0 .

Due to the convergence of sample representations from both views, the centroids and sample distributions of the views also tend to converge, i.e. lim ∥ ∥ µ (1) -µ ∗ ∥ ∥ = lim ∥ ∥ µ (2) -µ ∗ ∥ ∥ = 0 and lim ∥ ∥ q { 1 } -q ∗ ∥ ∥ = lim ∥ ∥ q { 2 } -q ∗ ∥ ∥ = 0 , where µ ∗ and q ∗ represent the clustering centroids and sample distributions respectively when the two views are perfectly aligned.

Therefore, the utility function based on distribution transportation tends to converge towards a same value, namely

According to the definition of Shapley values, the same utility function values yield the same Shapley values. Therefore, the normalized Shapley value | ϕ 1 -ϕ 2 |→ 0 .

A.2 Proof of Theorem 2

Theorem 2: For joint methods, considering the representation Z (1) = f (1) θ (1) ( X (1) ; θ (1) ) and Z (2) = f (2) θ (2) ( X (2) ; θ (2) ) on two views, with a simple view-level weight fusion during the fusion process: Z = w 1 Z (1) + w 2 Z (2) . There are situations where view 2 is dominated by view 1 and cannot make its contribution, i.e. , w 1 ≫ w 2 .

Proof: Suppose the representation of the two views:

and view-level weight fusion is adopted in the fusion process:

where w 1 and w 2 represent the weights of View 1 and View 2 respectively, satisfying w 1 + w 2 = 1 . Conduct Kmeans on the fused representation Z , the overall optimization goal is

where K is the number of clusters, C i is the sample set of the i -th cluster, and µ i is the centroid of C i , which can be represented as

where |C i | represent the number of samples in the sample set C i . For arbitrary sample z j ∈ C k j , the corresponding centroid is

The loss corresponding to the sample z j is

Derive with respect to w 1 for E j ,

The above only considered the individual sample z j to the total loss. When considering all samples

If N represents the total number of samples, taking the expectation Avg ( · ) of both sides of the equation yields

When ∂E ∂w 1 = 0 , the analytical solution for w 1 can be calculated, and at this point

where expression Avg ( ∥ z ( i ) j -µ ( i ) k j ∥ 2 2 ) reflects the level of difficulty in clustering for view i ( i ∈ { 1 , 2 } ); the clearer the clustering structure, the smaller the Avg ( ∥ z ( i ) j -µ ( i ) k j ∥ 2 2 ) .

then derive with respect to w 1 for E ,

And ( z (1) -µ (1) k j ) T ( z (2) j -µ (2) k j ) can be seen as a dot product similarity, reflecting the correlation between the two views; the smaller the correlation between views, the closer ( z (1) -µ (1) k j ) T ( z (2) j -µ (2) k j ) is to 0.

Assuming significant differences in clustering structures between two views, with view 1 being the dominant view over view 2, where Avg ( ∥ z (1) j -µ (1) k j ∥ 2 2 ) ≪ Avg ( ∥ z (2) j -µ (2) k j ∥ 2 2 ) and Avg ( ( z (1) -µ (1) k j ) T ( z (2) j -µ (2) k j ) ) ≈ 0 , then when the loss of multi-view clustering converges, the weights of the two views

which means in the fusion of views, view 1 takes the lead, while view 2 can only play a limited role. However, in unsupervised tasks, the clarity of the clustering structure is unrelated to the quality of the clustering results. Even though the clustering structure of view 2 may not be clear, the clustering results of view 2 could still be accurate. In this scenario, w 1 ≫ w 2 can be considered that the cooperation between the two views is suboptimal.

A.3 Proof of Theorem 3

Theorem 3: For joint methods that use view-level weight fusion Z = w 1 Z (1) + w 2 Z (2) , where Z (1) = f (1) θ (1) ( X (1) ; θ (1) ) and Z (2) = f (2) θ (2) ( X (2) ; θ (2) ) , gradient modulation in Eq. (18) allows the two views to contribute more evenly, i.e. , w 1 w 2 → 1 .

Proof: Introducing the suppression factor k (where k < 1 ) suppresses the convergence speed of the dominant view's parameters. In this case, the parameter updates become:

where the overall optimization objective for multi-view learning is given by

Due to the equivalence ∥ z -µ i ∥ 2 2 = ∥ ∥ ∥ w 1 ( z (1) j -µ (1) k j ) + w 2 ( z (2) j -µ (2) k j )∥ ∥ ∥ 2 2 , in accordance with the Cauchy-Schwarz inequality, we obtain:

Therefore, the upper bound on the overall loss E is jointly determined by the individual view losses and the view weights. As both ∥ z (1) j -µ (1) k j ∥ 2 2 and ∥ z (2) j -µ (2) k j ∥ 2 2 can be seen as the Kmeans loss for a single view, during one iteration, if we suppress the gradient descent for the dominant view (view 1), ∥ z (1) j -µ (1) k j ∥ 2 2 would increase compared to not suppressing it, while ∥ z (2) j -µ (2) k j ∥ 2 2 remains unaffected. The upper bound on the overall optimization objective shifts, leading to differences in the single-view parameters and weights at convergence.

As the loss converges, the ratio of weights between the two views, w 1 w 2 → 1 . This implies that the weight of the non-dominant view increases, allowing it to more fully engage in the fusion process. Consequently, there is improved cooperation between the views, enhancing the fusion process.

B Further Experiments

B.1 View Contribution Comparison w/o SCE

In this subsection, we conducted additional ablation experiments on the SCE module based on the DMJC method across six datasets. Table 5 illustrates the variations in view contributions before and after using SCE. The experimental findings further corroborate our conclusions:

· There are cases where certain views in some datasets do not effectively contribute, such as the 2 -nd view of Caltech101-7, the 2 -nd view of CUB, the 3 -rd and 4 -th views of STL10, and the 5 -th view of Reuters, all with contributions below 0.1.

· Through the SCE method, the contributions of suppressed views experienced significant growth, with the contribution of the 2 -nd view of Caltech101-7 increasing from 0.032 to 0.433, and the contribution of the 5 -th view of Reuters increasing from 0.107 to 0.232.

B.2 Detailed Discussions of View Roles

In this subsection, we conducted further exploration on the UCI-digit dataset to clarify the roles played by different views in cooperation and the effectiveness of cooperation enhancing module. In

Table 6, we illustrate the ACC of all views, single view, and paired views after training through three views of UCI-digit. Before we used the view cooperation enhancing module, the fusion results of View1 and View3 were superior to those of all views, although the ACC of View3 as a single view was worst amongst all. Therefore, we consider that views have shortcomings in cooperation. View1 dominates the fusion process, resulting in View2 and View3 not playing a better role. To this end, we used the view cooperation enhancing module to enhance the contributions of View2 and View3.

Furthermore, we compared the results in Table 7 when only two views participated in training. Although the fusion of View1 and View3 showed superior performance when three views participated in training, when training with only View1 and View3, the performance of fusion is significantly lower than that of training with three views. Besides, the fusion performance of paired views is not as good as that of three views, proving that each view in the dataset can play a positive role in cooperation, and this effect can be further expanded by enhancing cooperation.

B.3 Experiments on Additional Datasets

In this subsection, we conducted experiments on three additional datasets with two views: WikipediaArticles[54], SCENE15[55], and NoisyMNIST 8 . The detailed information about the datasets is shown in Table 8. Considering the scale of the NoisyMNIST dataset, all experiments were conducted on PyTorch platform using the NVIDIA 3090 GPU. The experimental results on these three datasets are presented in Table 9. The experimental results confirm our proposed theory and demonstrate the validity of our View Contribution Evaluation Module as well as the effectiveness of our View Cooperation Enhancing Module.

C Limitations and Future Work

In this section, we discussed the limitations of the SCE model and potential future work:

On some extreme datasets, certain view contributions obtained by the View Contribution Evaluation Module may be negative, which contradicts the non-negativity definition of Shapley values. A

negative Shapley value indicates that the corresponding view has performed poorly in the fusion process, raising the question of whether these views should be discarded, which could be a potential area of future research.

· The View Cooperation Enhancing Module narrows the contributions between views through gradient modulation, but it cannot extend the contributions between views. Whether there exists a more flexible method to control the contribution relationship between views is an intriguing question.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Please see the abstract and introduction sections.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please see the Appendix C.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Please see the Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please see the Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code has been open access.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please see the Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: The paper does not report error bars.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please see the Section 4.1.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
This paper studies multi-view clustering and seeks to investigate the view cooperation issue. The authors consider DMVC as an unsupervised cooperative game and regard each view as a participant. Compared with the existing methods, this consideration is new and interesting. Based on the novel idea, the authors proposed SCE-MVC, a novel shapley-based cooperation enhancing multi-view clustering method. The paper is well-organized. The experiments are convincing.

STRENGTHS:
1. The paper proposes a new point also an interesting point for multi-view clustering tasks, i.e., considering the multi-view collaboration as a cooperative game. 

2. The experiments are sufficient and convincing. The authors validate the method from many aspects. The proposed SCE-MVC obtains much better performance on six diverse datasets.

WEAKNESSES:
1. Figure 2 is confusing. The specific structure of View Cooperation Enhancing Module is not clearly presented.

2. There are many formulas and symbols. It is suggested to add a notation table.

3. Although the authors try to explain model (1), it is still difficult to understand Shapley Value from the model. In addition, many variables are not clearly explained. The authors should present more information about the model and explain all variables used in this model, such as S_i, {i}, s\{i}， etc.

QUESTIONS:
The article involves rich theoretical and mathematical knowledge. I have a question to the designation of the model: Which design is the key to improving model performance?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2024-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author introduces a Shapley-based cooperation enhancement framework aimed at fostering collaboration among different views. The SCE-MVC method incorporates cooperative game theory, considering each view as a participant in the model and assessing their contributions using the Shapley Value.\n\nSTRENGTHS:\nViewing each view as an individual player within game theory represents a fresh perspective in multi-view clustering. Also, enhancing clustering performance through balancing view contribution is both well-founded and innovative.\n\nWEAKNESSES:\n1. Using the SCE module in an alignment-based framework only provides a marginal improvement to the model. Does this imply that the SCE module is ineffective in the alignment-based framework? \n\n2. The view contributions of alignment-based method is much balanced than view contributions of joint methods. Does this imply that the alignment-based method is much better than the joint method? It's not reasonable since the clustering performance of alignment-based methods may not necessarily be better than that of joint methods.\n\n3. Is the complexity of computing Shapley values truly O(n!)? When dealing with a larger number of views, can this evaluation framework still be utilized for computation?\n\n4. Are the loss functions L in Eqs (15) and (16) on page 6 the same? If so, there is a problem of inconsistent dependent variables. In addition, $D_ij$ in Eq. (9) is a scalar and should not be bolded.\n\nQUESTIONS:\nThe alignment-based method proposed in Theorem 1 will make the contribution values of several views the same. Combined with the experimental results in Table 2, does this mean that the end point of the view contribution optimization proposed in this paper is contrastive learning? If not, please explain in detail the difference between the method in this paper and the contrastive learning method?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

comments: SUMMARY:
The study centers on improving task performance via deep multi-view clustering (DMVC) and fostering cooperation among different views. Specifically, the study evaluates view contributions, emphasizing the significance of strengthening cooperation among views.

STRENGTHS:
Considering multi-view tasks from a collaborative standpoint represents a novel approach, with the paper's motivation being notably fresh. Moreover, the paper elucidates potential contribution imbalances in the joint method and addresses them through the SCE method, thereby enhancing cooperation among views.

WEAKNESSES:
When dealing with datasets comprising more than two views, such as three views, how can one assess whether the contribution of the views has become more evenly distributed after employing SCE? While the paper visually presents the contributions of the views, could a quantitative method be provided for this evaluation?

QUESTIONS:
In the unsupervised multi-view scenario, what is the physical meaning of the contribution value of each view proposed in this paper? What is the relationship between the quantitative value of the view's contribution and the clustering performance of a single view ?

LIMITATIONS:
The authors have adequately addressed the limitations and potential negative societal impact of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

comments: SUMMARY:
This research merges game theory with multi-view clustering by introducing the Shapley-based Cooperation Enhancing (SCE) approach. It features a module to systematically evaluate each view's contribution. The approach promotes view cooperation by adjusting the training convergence rate of view parameters based on their contributions. Extensive experiments on various datasets demonstrate the method's effectiveness when applied to different MVC frameworks.

STRENGTHS:
1) The paper integrates the Shapley value from game theory into DMVC, allowing for precise assessment of each view's contribution.
2) Theoretical analysis is thorough, with clear and intuitive figures.
3) The manuscript is well-organized and clearly written.

WEAKNESSES:
The article categorizes DMVC into alignment-based and joint methods. What criteria were used for this classification? Furthermore, only one DMJC method is used as a representative for joint methods.

QUESTIONS:
1) Figure 3(a) indicates that the method does not equalize the contribution value of each view. Why do the contribution values become identical after adding the SCE module to the comparison-based method in Table 2? Please provide a detailed discussion.
2) What criteria were used to classify DMVC?
3) Is DMJC representative of joint methods? Have other joint methods employed similar frameworks?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

comments: SUMMARY:
This paper firstly considered DMVC as an unsupervised cooperative game where each view can be regarded as a participant. Then, the authors introduced the shapley value and propose a novel MVC framework termed Shapley-based Cooperation Enhancing Multi-view Clustering (SCE-MVC), which evaluates view cooperation with game theory. In summary, this paper was well written with obvious superiority.

STRENGTHS:
-- A MVC framework was designed that utilizeD game theory and Shapley values to evaluate and elevate inter-view cooperation. 
-- The experiments were sufficient, and the analysis of the experimental results was adequate.

WEAKNESSES:
-- In this paper, why utilize $\phi_i$ to measure the contribution of views instead of the view weight $w_i$? The article's explanation on this is not clear enough, and there is a lack of experiments to demonstrate the relationship between $\phi_i$ and $w_i$.

QUESTIONS:
I have the following questions:
-- What will happen if the view contribution is push away from each other? 
-- Are there scenarios where narrowing the contribution between views fails to enhance the effectiveness of multi-view clustering?

LIMITATIONS:
The authors have adequately addressed the limitations and potential negative societal impact of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-12