Evaluate then Cooperate: Shapley-based View Cooperation Enhancement for Multi-view Clustering

Authors: fangdi  wang
jiaqi  jin
jingtao  hu
suyuan  liu
xihong  yang
siwei  wang
xinwang  liu
en  zhu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-26 00:31:18

Abstract: The fundamental goal of deep multi-view clustering is to achieve preferable task performance through inter-view cooperation. Although numerous DMVC approaches have been proposed, the collaboration role of individual views have not been well investigated in existing literature. Moreover, how to further enhance view cooperation for better fusion still needs to be explored. In this paper, we firstly consider DMVC as an unsupervised cooperative game where each view can be regarded as a participant. Then, we introduce the Shapley value and propose a novel MVC framework termed Shapley-based Cooperation Enhancing Multi-view Clustering (SCE-MVC), which evaluates view cooperation with game theory. Specially,  we employ the optimal transport distance between fused cluster distributions and single view component as the utility function for computing shapley values. Afterwards, we apply shapley values to assess the contribution of each view and utilize these contributions to promote view cooperation. Comprehensive experimental results well support the effectiveness of our framework adopting to existing DMVC frameworks, demonstrating the importance and necessity of enhancing the cooperation among views.

Main paper: 
 introduction: 
 Recently, multi-view clustering has become one of the most prominent problems in unsupervised learning [1, 2, 3, 4] . It leverages data from different views, combining them to provide richer information for clustering tasks. Generally speaking, multi-view clustering can be broadly categorized into traditional methods [5, 6, 7, 8] and deep multi-view clustering (DMVC) [5, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] methods. Compared to the limited feature extraction capabilities of traditional methods, deep multi-view clustering methods better preserve information from different views through flexible deep neural networks. For most DMVC methods, the key challenge is how to enhance cooperation among views to accomplish better clustering performance [20] .Numerous DMVC algorithms have already achieved significant results in downstream clustering tasks. However, there has been little research on investigating the the contribution of each view in the fusion stage. Once evaluating the view contribution, we may observe the phenomenon: One view dominates the fusion process, suppressing the collaborative contribution of other views, leading to suboptimal clustering results. We attribute this phenomenon to insufficient cooperation between views, and thus propose a fundamental assumption: More balanced contributions and more extensive cooperation among multiple views can lead to better clustering results, as shown in Fig. 1 . To this end, we consider introducing the concept of game theory into multi-view clustering and propose the Shapley-based Cooperation Enhancing (SCE) method. Shapley value [21, 22] is a concept used to measure the contribution of participants in game theory [23, 24] . It utilizes a utility function to evaluate the different marginal contributions that each participant brings when combined with other participants, and calculates their contribution to the overall cooperation by taking the weighted average over all possible combinations of participants [21] . Upon quantifying the contribution of each view, the cooperation among views can be analyzed based on their respective contributions. By employing a module that enhances view cooperation, we can dynamically adjust the parameter convergence rate of views during training in accordance with their contributions. This enables the underrepresented views to receive a more balanced training regimen, thereby enhancing post-fusion performance [25] .The primary contributions of this work can be summarized as follows:• Our work is the first framework to evaluate the view contributions within an unsupervised context. Leveraging the proposed View Contribution Evaluation Module, we theoretically quantify the individual contributions of views.• We propose the View Cooperation Enhancing Module based on the view contributions obtained by the View Contribution Evaluation Module, allowing suppressed views to effectively participate in the fusion process, ultimately enhancing clustering performance.• Comprehensive experiments have been conducted to showcase the versatility of our SCE framework, which can effectively evaluate the contributions of views across various MVC frameworks.2 Related WorkQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: Please see the abstract and introduction sections.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
deep multi-view clustering: 
 In deep multi-view clustering [26, 27, 28, 29] , deep neural networks [30, 31] with multiple nonlinear transformations are more effective at acquiring feature representations than traditional shallow models. Presently, deep multi-view clustering methods fall into three main categories [32, 33] : joint methods [34, 35, 36] , alignment-based methods [14, 37, 38, 39] , and other methods [40, 41, 42] . Joint methods consider differences and complementarities among views, optimizing sample representations within each view's space. For example, the DMJC [35] utilizes autoencoders to generate view representations, refining them as pseudo-labels for self-supervised optimization. Alignment-based methods focus on consistency between views, mapping representations into a shared subspace. These methods often employ contrastive learning, such as ProImp [43] , which integrates contrastive learning at both sample and prototype levels. Some approaches combine joint and alignment-based methods, like the share-specific method [40] , which separates sample representations into shared and specific components, aligns shared representations into a common space, and refines specific representations in separate spaces.The core concept of DMVC is to enhance clustering performance by fostering enhanced cooperation and utilizing complementary information across views. Although existing DMVC research emphasizes leveraging inter-view cooperation for improved clustering tasks, there is currently a lack of research quantifying the contribution of views during the fusion stage to enhance cooperation.Multi views [20] and multi modalities [44] are two closely intertwined domains. Within the realm of multimodal fusion, studies have surfaced highlighting the issue of subpar collaboration among modalities, prompting endeavors to alleviate the imbalance in modality cooperation. Wang et al. [45] uncovered varying convergence rates among diverse modalities, leading to a scenario where jointly trained multimodal models struggle to match or outperform their unimodal counterparts. Furthermore, Peng et al. [25] demonstrated that in the context of audio-visual learning, superior performance in the audio modality hampers the optimization of the video modality.Drawing inspiration from prior studies, our research aims to evaluate the individual contributions of each view in multi-view clustering fusion. Nevertheless, evaluating view contribution based solely on labels in unsupervised environments is infeasible. Moreover, employing weights as a measure for view contribution may not align with all DMVC structures, especially those based on contrastive learning frameworks. Consequently, quantifying view contributions in fusion and fostering their cooperation pose significant research challenges.In order to reasonably and effectively evaluate the contributions of each view in the fusion process, we refer to the theoretical framework of game theory. The Shapley value [46] within game theory proves instrumental in quantifying participants' contributions to cooperative coalitions. Denote X = {x i } n i=1 as the alliances of n participants. For a participant x i , the Shapley value considers x i 's contribution in every set that includes him. Let S i = {S ⊆ X |x i ∈ S} represent the set of all subsets of X that include the participant x i , then, the overall contribution of member x i , i.e., the Shapley value, can be expressed asEQUATIONwhere |s| signifies the cardinality of set s; v(•) represents the utility function; [υ(s) -υ(s\{i})] quantifies the marginal contribution of x i to set s; and (|s|-1)!(n-|s|)! n! denotes the weight assigned to this marginal contribution, determined by the probability of set s occurring.Given the advantageous characteristics of Shapley values-efficiency, symmetry, dummy, and additivity-researchers have increasingly employed them for explaining machine learning models. For instance, Lundberg et al. [47] proposed a method to interpret predictions by computing feature importance based on Shapley values. Hu et al. [48] applied Shapley values to evaluate modality contributions in supervised tasks, while Wei et al. [49] explored modality assessment using Shapley values. However, these studies focus mainly on supervised scenarios, making the definition of utility functions for Shapley values in unsupervised settings a challenging task yet to be resolved.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Please see the Appendix C.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
problem statement: 
 Given a multi-view dataset X = {X (1) , X (2) , ..., X (V ) } , where V is the number of views, and X (v) (v = 1, 2, ..., V ) denotes the original feature space of the v-th view. Consider the clustering problem on the dataset, each cluster is represented by a clustering center µ (v) j , j = 1, ..., K, where K is the number of clusters. To achieve better clustering performance, we map the original feature space X (v) to a latent embedded feature space Z (v) through a non-linear mapping function Figure 2 : The framework of our proposed SCE model. SCE framework can be applied to most mainstream DMVC methods. After obtaining the cluster distribution from the selected method, SCE iteratively using the following two modules: View Contribution Evaluation Module and View Cooperation Enhancing Module. View Contribution Evaluation Module computes the marginal contribution of views in each combination(Shapley value), in which the optimal transport distance between the view cluster distribution and the strengthened cluster distribution after fusion serves as the utility function of Shapley formula. View Cooperation Enhancing Module controls the convergence speed ratio of different views through the calculated view contribution. v) , where θ (v) represents the learnable parameters of mapping function for the v-th view.f (v) θ (υ) : X (v) → Z (To assess the contributions of each view in MVC tasks, we treat each view as a player and introduce the Shapley value referring to Eq. (1) . Let X = {X (i) } V i=1 be the set of all views, S v = {S ⊆ X |X (v) ∈ S} represent the set of all subsets of X that include the v-th view X (v) . Define an internal metric of clustering E independently of labels to serve as a utility function, which will be elaborated on in the next subsection. Then the Shapley value of the v-th view can be calculated asEQUATIONwhere |s| denotes the number of views in the view-set s.In Shapley value, the utility function is used to evaluate the benefit created by a coalition. In DMVC, where each view acts as a participant, the utility function needs to be designed to reflect the contribution of any combination of views in fusion, without the need for label assistance. In this scenario, we constructed a novel function E using the following method.Inspired by the DEC model [50] , we assume that the data from each view follows a Student's tdistribution. Following the approach outlined by [51] , we utilize the t-distribution as a kernel function to evaluate the distance between the sample embeddings z i and the cluster centers µ j within a view:EQUATIONwhere α represents the degrees of freedom in the t-distribution. q ij denotes the probability of assigning the feature z i to the cluster center µ j .Fix the degrees of freedom α to 1, the cluster distribution of multi-view data q can be considered as a combination of single-view cluster distribution based on a weight matrix π:EQUATIONwhere the weight matrix π ∈ R K×V represents the importance of the cluster centers µ j in different views. This weight matrix π can be obtained by performing row normalization on an unconstrained trainable matrix W:EQUATIONFor any given combination of an m-view union U = {X (u1) , X (u2) , ..., X (um) }, we can obtain the weight matrix π U by selecting the corresponding columns in the matrix W. By applying Eq. ( 3), we derive the data distribution for the fused view, which is denoted as q U .In particular, if U = ∅, which means no view is involved in the clustering process, we consider q ∅ to be a uniform distribution. This means that the probability of any embedding z i belonging to any cluster µ j is equal and given by 1/K, where K is the total cluster numbers. In this case, we denote the resulting data distribution q ∅ asEQUATIONwhere 1 N ×K ∈ R N ×K is a matrix in which all elements are equal to 1.According to the principle of self-distillation in DEC, we sharpen the fused distribution q X from all views to obtain a more powerful target distribution p:EQUATIONSince we consider the distribution p as the trend of the fusion process, the closer the fusion distribution q U (combined with U = {X (u1) , X (u2) , ..., X (um) }) is to the distribution p, the higher the participation of view set U in the fusion process. Therefore, we can use the optimal transport distance from distribution q U to distribution p as a measure of confidence for distribution q U . Additionally, based on our desire for the metric to possess monotonicity and boundedness, we define the utility function based on distribution transportation as:EQUATIONwhere OT (•, •) is calculated as wasserstein distance and E(•) ∈ (0, 1]. The closer the data distribution is to the pseudo-labels p, the larger the value of the metric E(•).In the process of computing the optimal transport OT (•, •), we consider the distribution q as a probability set where K cluster centers are assigned by N samples. The probability of each cluster center is 1/K. For example, when calculating OT (q {X (1) } , q {X (2) } ), the distribution of views can be represented as [1/K, 1/K, ..., 1/K] T , and the distance matrix {D ij } K×K can be calculated asEQUATIONwhere the j-th cluster center for the v-th view can be represented as ξ(v) j = {q (v) ij } N ×1 .In the above subsections, we have already discussed the integration of cooperative game theory with multi-view clustering tasks. We have also defined a utility function E that reflects the accuracy of clustering. Now, combining the Eq. ( 2) and ( 8), we present the formal representation of viewcontribution through Shapley values:EQUATION)The sum of all view contributions is then calculated as:EQUATIONAs p and q X evolve throughout training, the sum of Shapley values for all views also fluctuate accordingly. To obtain the contribution proportions of each view to the fused view, letEQUATIONwhere φ v is the normalized Shapley value, determining the respective contribution of different views.To further demonstrate the generalization performance of the View Contribution Evaluation Module, we apply it to the alignment-based method and the joint method, respectively. The relevant theoretical derivations are given below.Theorem 1: For alignment-based methods, considering the representation {zi } N i=1 and {z(2) i } N i=1on two views, with the infoNCE contrastive learning loss:EQUATION(1)EQUATIONEQUATIONEQUATION)where N is the sample number and τ > 0 is a scalar temperature hyperparameter. ℓ con (1, 2) brings the representations between views closer, leads to the average of view contributions, i.e., |ϕ 1 -ϕ 2 | → 0.Theorem 2: For joint methods, considering the representationEQUATIONθ (1) (X (1) ; θ (1) ) and (2) ; θ (2) ) on two views, with a simple view-level weight fusion during the fusion process: Z = w 1 Z (1) + w 2 Z (2) . There are situations where view 2 is dominated by view 1 and cannot make its contribution, i.e., w 1 ≫ w 2 .Z (2) = f (2) θ (2) (XRemark: Theorem 1 indicates that alignment-based methods tend to align the contributions of different views. Under our assumption, the cooperation between views is already sufficient, optimizing the model based on contributions can only lead to limited improvements. Theorem 2 suggests that in joint methods, one view may suppress another, highlighting insufficient cooperation between views. In such cases, enhancing cooperation between views theoretically improves model performance. We will elaborate on the View Cooperation Enhancing Module in Section 3.4 and validate these conclusions through experiments. The proofs for Theorems 1 and 2 can be found in Appendix.After acquiring contributions of different views, we seek to enhance their cooperation by dynamically regulating views' training speeds according to these contributions. This proportional control guarantees active involvement of all views in the integration process. For this purpose, we introduce the convergence speed ratio, labeled as k, derived from the contributions of each view.To ensure the applicability of our Shapley-based Cooperation Enhancing(SCE) method to most multi-view clustering tasks, we consider a typical multi-view clustering framework: firstly, obtain embeddings for each view by utilizing autoencoders; then, fuse the embeddings from different views using a global optimization goal. Let f(v) θ (v) (X (v) ; θ (v)) represent the encoder model for the v-th view, g denote the fusion method for multiple views, and l represent the loss function for computing the fusion loss. The overall fusion loss L can be represented asEQUATIONIn practice, the parameters for the v-th view is updated asEQUATIONWith the normalized Shapley values calculated by Eq. ( 12) to dynamically monitor the contribution differences between different views, we are able to adaptively modulate the gradients via an empirical formula:EQUATION)where τ is a temperature hyperparameter to control the degree of modulation. Thus k v is not greater than 1, and the larger the contribution of the view, the smaller the k v . We integrate the coefficient k i into the Adam optimization method, and the update of θ(v)t+1 at iteration t is as follows:EQUATIONwhere k v is used as a relative convergence speed ratio for the v-th view in gradient descent.Theorem 3: For joint methods that use view-level weight fusion Z = w 1 Z (1) + w 2 Z (2) , whereEQUATIONθ (1) (X (1) ; θ (1) ) and 2) ; θ (2) ), gradient modulation in Eq. ( 18) allows the two views to contribute more evenly, i.e., w1 w2 → 1. Remark: Theorem 3 indicates that through dynamic gradient adjustments, we can harmonize the learning progress among different views, attain a more average distribution of contributions, and enhance the cooperation among views. The proof for Theorem 3 can be found in Appendix. 4), ( 7) and (10) ; Normalize the shapley value as view contribution with Eq. ( 12); // View Cooperation Enhancing Calculate the convergence speed ratio k v for each view with Eq. ( 17); Update the parameters of each view relatively with Eq. ( 18); return;EQUATIONInput: Multi-view dataset {X (v) } V v=1 ,This subsection outlines the implementation of our SCE module, detailed in Algorithm 1. After obtaining initial cluster center assignments, we iteratively employ the following two modules: View Contribution Evaluation. First, we calculate the cluster distribution for each individual view as well as the combined view. Next, we compute an enhanced distribution p. After utilizing the optimal transport distance between distributions as the utility function, we can calculate view contributions.View Cooperation Enhancing. We utilize the calculated contribution values to influence the training process of each view using a convergence ratio k. The larger the contribution value of a view, the less the convergence ratio k will be. This will coordinate the training process of the views, enabling better cooperation among views, and allowing them to collaborate more effectively.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
experiments: 
 In this section, we implement experiments on alignment-based method and joint method to verify the effectiveness of the proposed theorems and SCE module by addressing the following questions: Our experiments utilized six multi-view datasets, including CUB2 , Caltech101-7 3 , HandWritten4 , UCI-digit5 , STL10 6 and Reuters 7 . The detailed information of these datasets is listed in the Table 1 .Meanwhile, three widely used metrics are adopted in our experiment, including clustering accuracy (ACC), normalized mutual information (NMI) and adjusted rand index(ARI).For fairness, We conduct all experiments on PyTorch platform using the NVIDIA 2060 GPU. Besides, ten state-of-the-art MVC methods are introduced: DEMVC[13], CoMVC [14] , SiMVC [14] , SDSNE [15] , MFLVC [16] , SDMVC [17] , DSMVC [18] , APADC [52] , DMJC [35] and ProImp [43] .In this section, we apply the SCE methods to two alignment-based MVC methods:-Utilizing infoNCE as a contrastive loss and employing Kmeans after concatenation.-Incorporating contrastive learning at both the sample and prototype levels as ProIMP [43] .We applied these two approaches to the CUB and Caltech101-7 datasets, each comprising two views. Detailed experimental results are presented in the Table 2 . The results lead to following conclusions: • Assessed with the View Contribution Evaluation Module, the contributions of the two views in alignment-based methods are remarkably similar, validating the assertion of Theorem 1.• As the contributions of the views tend to equalize, the utilization of the View Cooperation Enhancing Module approach only yields subtle improvements to the alignment-based methods.In this subsection, we apply the SCE model to a joint MVC methods(DMJC [35] ). We have carried out comprehensive experiments on six datasets with varying numbers of views, as depicted in Table 3 and Fig. 3 . In Table 3 , we showcase the performance of SCE-MVC in comparison to ten benchmark methods. Fig. 3 illustrates the ablation study of DMJC with and without SCE. Based on the aforementioned experimental results, we draw the following conclusions:• Fig. 3 (a) illustrates the changes in view contributions with and without the SCE module across three datasets. Prior to the use of the SCE module, there is a scenario where certain views dominate while others are suppressed, a pattern particularly evident in the Caltech101-7 dataset, aligning well with the conclusions of Theorem 2.• After introducing the View Contribution Enhancing Module, the contributions of the views become more balanced, leading to a significant improvement in the model's performance. Specifically, the ACC, NMI, and ARI metrics on the Caltech101-7 dataset all witness an increase of over 10 percentage points, thus validating the conclusion of Theorem 3. More detailed contribution comparison data can be found in the Appendix. In SCE-MVC, the singular hyperparameter τ is essential for fine-tuning the convergence equilibrium among distinct views according to their contributions. In sensitive analysis, we varied τ within {0.5, 1, 1.5, 2, 2.5, 3}, as illustrated in Table 4 . As τ increases, the changes in clustering results metrics become less pronounced, which can be attributed to the characteristics of the exponential functions in Eq. (17) . Specifically, as τ increases, the convergence ratio k v tends to be closer to zero. 
conclusion: 
 This paper explores the roles of views in multi-view clustering through the lens of game theory. We introduce a novel contribution evaluation module founded on the Shapley value. Our experiments showcase the adaptability of this module across mainstream MVC frameworks, accurately assessing each view's impact. By scrutinizing the functions of diverse views during fusion, we devise a cooperation enhancing module to bolster cooperation among views. Extensive dataset validations underscore the effectiveness and robust applicability of our approach and theorems. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA]Justification: The paper does not use existing assets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Proof of Theorem 1Theorem 1: For alignment-based methods, considering the representation {zi } N i=1 and {z(2) i } N i=1on two views, with the infoNCE contrastive learning loss:EQUATIONj /τ l ) + exp(zEQUATIONEQUATION)where N is the sample number and τ > 0 is a scalar temperature hyperparameter. ℓ con (1, 2) brings the representations between views closer, leads to the average of view contributions, i.e., |ϕ 1 -ϕ 2 | → 0.Proof: Referring to Definition of Perfect Alignment in [53] , we say the two views are perfectly aligned if zEQUATIONi , i = 1, 2, ..., N . Assuming that with the training of contrastive learning, the representations of the two views tend to align perfectly. Then, there exists a limiting representationZ * such that lim ∥Z 1 -Z * ∥ = lim ∥Z 2 -Z * ∥ = 0.Due to the convergence of sample representations from both views, the centroids and sample distributions of the views also tend to converge, i.e. lim µ (1) µ * = lim µ (2) -µ * = 0 and lim q {1} -q * = lim q {2} -q * = 0, where µ * and q * represent the clustering centroids and sample distributions respectively when the two views are perfectly aligned.Therefore, the utility function based on distribution transportation tends to converge towards a same value, namelyEQUATIONAccording to the definition of Shapley values, the same utility function values yield the same Shapley values. Therefore, the normalized Shapley value |ϕ 1 -ϕ 2 | → 0.Theorem 2: For joint methods, considering the representationEQUATIONθ (1) (X (1) ; θ (1) ) and Z (2) = f(2) θ (2) (X (2) ; θ (2) ) on two views, with a simple view-level weight fusion during the fusion process: Z = w 1 Z (1) + w 2 Z (2) . There are situations where view 2 is dominated by view 1 and cannot make its contribution, i.e., w 1 ≫ w 2 .Proof: Suppose the representation of the two views:Z (1) = f (1)θ (1) (X (1) ; θ (1) ), (2) ; θ (2) ), (22) and view-level weight fusion is adopted in the fusion process:Z (2) = f (2) θ (2) (XEQUATIONwhere w 1 and w 2 represent the weights of View 1 and View 2 respectively, satisfying w 1 + w 2 = 1.Conduct Kmeans on the fused representation Z, the overall optimization goal isEQUATION)where K is the number of clusters, C i is the sample set of the i-th cluster, and µ i is the centroid of C i , which can be represented asEQUATIONwhere |C i | represent the number of samples in the sample set C i .For arbitrary sample z j ∈ C kj , the corresponding centroid isEQUATIONThe loss corresponding to the sample z j isEQUATION)Derive with respect to w 1 for E j ,EQUATION)The above only considered the individual sample z j to the total loss. When considering all samplesEQUATIONthen derive with respect to w 1 for E,EQUATION)If N represents the total number of samples, taking the expectation Avg(•) of both sides of the equation yieldsEQUATION)When ∂E ∂w1 = 0, the analytical solution for w 1 can be calculated, and at this pointEQUATION)where expression Avg(∥zEQUATION2 ) reflects the level of difficulty in clustering for view i (i ∈ {1, 2}); the clearer the clustering structure, the smaller the Avg(∥z(i) j -µ (i) kj ∥ 2 2 ).And z (1) µ(1) kj T z(2) j -µ(2) kj can be seen as a dot product similarity, reflecting the correlation between the two views; the smaller the correlation between views, the closerz (1) -µ (1) kj T z (2) j -µ(2) kj is to 0. Assuming significant differences in clustering structures between two views, with view 1 being the dominant view over view 2, where Avg(∥zEQUATIONkj ) ≈ 0, then when the loss of multi-view clustering converges, the weights of the two viewsw 1 w 2 ≈ Avg∥z (2) j -µ (2) kj ∥ 2 2(1)EQUATIONwhich means in the fusion of views, view 1 takes the lead, while view 2 can only play a limited role. However, in unsupervised tasks, the clarity of the clustering structure is unrelated to the quality of the clustering results. Even though the clustering structure of view 2 may not be clear, the clustering results of view 2 could still be accurate. In this scenario, w 1 ≫ w 2 can be considered that the cooperation between the two views is suboptimal.Theorem 3: For joint methods that use view-level weight fusion Z = w 1 Z (1) + w 2 Z (2) , whereEQUATIONθ (1) (X (1) ; θ (1) ) and 2) ; θ (2) ), gradient modulation in Eq. ( 18) allows the two views to contribute more evenly, i.e., w1 w2 → 1. Proof: Introducing the suppression factor k (where k < 1) suppresses the convergence speed of the dominant view's parameters. In this case, the parameter updates become:EQUATIONθ (1) = θ (1) -k • η ∂E ∂θ (1) θ (2) = θ (2) -η ∂E ∂θ (2) (34)where the overall optimization objective for multi-view learning is given byEQUATION)Due to the equivalence ∥z -µ i ∥ 2 2 = w 1 z (1) j -µ (1) kj + w 2 z (2) j -µ (2) kj 2 2, in accordance with the Cauchy-Schwarz inequality, we obtain:EQUATION)Therefore, the upper bound on the overall loss E is jointly determined by the individual view losses and the view weights. As both ∥z(1)j -µ (1) kj ∥ 2 2 and ∥z (2) j -µ (2)kj ∥ 2 2 can be seen as the Kmeans loss for a single view, during one iteration, if we suppress the gradient descent for the dominant view (view 1), ∥z(1) j -µ (1) kj ∥ 22 would increase compared to not suppressing it, while ∥z(2)j -µ (2)kj ∥ 2 2 remains unaffected. The upper bound on the overall optimization objective shifts, leading to differences in the single-view parameters and weights at convergence.As the loss converges, the ratio of weights between the two views, w1 w2 → 1. This implies that the weight of the non-dominant view increases, allowing it to more fully engage in the fusion process. Consequently, there is improved cooperation between the views, enhancing the fusion process.In this subsection, we conducted additional ablation experiments on the SCE module based on the DMJC method across six datasets. Table 5 illustrates the variations in view contributions before and after using SCE. The experimental findings further corroborate our conclusions:• There are cases where certain views in some datasets do not effectively contribute, such as the 2-nd view of Caltech101-7, the 2-nd view of CUB, the 3-rd and 4-th views of STL10, and the 5-th view of Reuters, all with contributions below 0.1.• Through the SCE method, the contributions of suppressed views experienced significant growth, with the contribution of the 2-nd view of Caltech101-7 increasing from 0.032 to 0.433, and the contribution of the 5-th view of Reuters increasing from 0.107 to 0.232. In this subsection, we conducted further exploration on the UCI-digit dataset to clarify the roles played by different views in cooperation and the effectiveness of cooperation enhancing module. In 6 , we illustrate the ACC of all views, single view, and paired views after training through three views of UCI-digit. Before we used the view cooperation enhancing module, the fusion results of View1 and View3 were superior to those of all views, although the ACC of View3 as a single view was worst amongst all. Therefore, we consider that views have shortcomings in cooperation. View1 dominates the fusion process, resulting in View2 and View3 not playing a better role. To this end, we used the view cooperation enhancing module to enhance the contributions of View2 and View3.Furthermore, we compared the results in Table 7 when only two views participated in training.Although the fusion of View1 and View3 showed superior performance when three views participated in training, when training with only View1 and View3, the performance of fusion is significantly lower than that of training with three views. Besides, the fusion performance of paired views is not as good as that of three views, proving that each view in the dataset can play a positive role in cooperation, and this effect can be further expanded by enhancing cooperation.In this subsection, we conducted experiments on three additional datasets with two views: WikipediaArticles [54] , SCENE15 [55] , and NoisyMNIST8 . The detailed information about the datasets is shown in Table 8 . Considering the scale of the NoisyMNIST dataset, all experiments were conducted on PyTorch platform using the NVIDIA 3090 GPU. The experimental results on these three datasets are presented in Table 9 . The experimental results confirm our proposed theory and demonstrate the validity of our View Contribution Evaluation Module as well as the effectiveness of our View Cooperation Enhancing Module.In this section, we discussed the limitations of the SCE model and potential future work:• On some extreme datasets, certain view contributions obtained by the View Contribution Evaluation Module may be negative, which contradicts the non-negativity definition of Shapley values. A negative Shapley value indicates that the corresponding view has performed poorly in the fusion process, raising the question of whether these views should be discarded, which could be a potential area of future research.• The View Cooperation Enhancing Module narrows the contributions between views through gradient modulation, but it cannot extend the contributions between views. Whether there exists a more flexible method to control the contribution relationship between views is an intriguing question.NeurIPS Paper ChecklistJustification: There is no societal impact of the work performed.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).http://www.vision.caltech.edu/visipedia/CUB-200.htmlhttps://data.caltech.edu/records/mzrjq-6wc02https://archive.ics.uci.edu/ml/datasets/Multiple+Featureshttps://cs.nyu.edu/âĹĳroweis/data.htmlhttps://cs.stanford.edu/~acoates/stl10/http://archive.ics.uci.edu/ml/datasets/Reuters-21578+Text+Categorization+ Collectionhttp://yann.lecun.com/exdb/mnist/ 


Decision: This paper treats DMVC as an unsupervised cooperative game, with each view acting as a participant. The authors introduce the Shapley value and propose a novel MVC framework that assesses view cooperation using game theory. Extensive experiments validate the effectiveness of the proposed framework. While some potential issues remain, all reviewers acknowledge the contributions of this paper and have recommended it for acceptance.

Reviews: review: paper_summary: paper_summary: This paper firstly considered DMVC as an unsupervised cooperative game where each view can be regarded as a participant. Then, the authors introduced the shapley value and propose a novel MVC framework termed Shapley-based Cooperation Enhancing Multi-view Clustering (SCE-MVC), which evaluates view cooperation with game theory. In summary, this paper was well written with obvious superiority.
strength_weakness: strength_weakness: -- In this paper, why utilize $\phi_i$ to measure the contribution of views instead of the view weight $w_i$? The article's explanation on this is not clear enough, and there is a lack of experiments to demonstrate the relationship between $\phi_i$ and $w_i$.
questions: questions: I have the following questions:
-- What will happen if the view contribution is push away from each other? 
-- Are there scenarios where narrowing the contribution between views fails to enhance the effectiveness of multi-view clustering?
limitations: limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work.
paper_summary: paper_summary: This research merges game theory with multi-view clustering by introducing the Shapley-based Cooperation Enhancing (SCE) approach. It features a module to systematically evaluate each view's contribution. The approach promotes view cooperation by adjusting the training convergence rate of view parameters based on their contributions. Extensive experiments on various datasets demonstrate the method's effectiveness when applied to different MVC frameworks.
strength_weakness: strength_weakness: The article categorizes DMVC into alignment-based and joint methods. What criteria were used for this classification? Furthermore, only one DMJC method is used as a representative for joint methods.
questions: questions: 1) Figure 3(a) indicates that the method does not equalize the contribution value of each view. Why do the contribution values become identical after adding the SCE module to the comparison-based method in Table 2? Please provide a detailed discussion.
2) What criteria were used to classify DMVC?
3) Is DMJC representative of joint methods? Have other joint methods employed similar frameworks?
limitations: limitations: Yes
paper_summary: paper_summary: The study centers on improving task performance via deep multi-view clustering (DMVC) and fostering cooperation among different views. Specifically, the study evaluates view contributions, emphasizing the significance of strengthening cooperation among views.
strength_weakness: strength_weakness: When dealing with datasets comprising more than two views, such as three views, how can one assess whether the contribution of the views has become more evenly distributed after employing SCE? While the paper visually presents the contributions of the views, could a quantitative method be provided for this evaluation?
questions: questions: In the unsupervised multi-view scenario, what is the physical meaning of the contribution value of each view proposed in this paper? What is the relationship between the quantitative value of the view's contribution and the clustering performance of a single view ?
limitations: limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work.
paper_summary: paper_summary: The author introduces a Shapley-based cooperation enhancement framework aimed at fostering collaboration among different views. The SCE-MVC method incorporates cooperative game theory, considering each view as a participant in the model and assessing their contributions using the Shapley Value.
strength_weakness: strength_weakness: 1. Using the SCE module in an alignment-based framework only provides a marginal improvement to the model. Does this imply that the SCE module is ineffective in the alignment-based framework? 

2. The view contributions of alignment-based method is much balanced than view contributions of joint methods. Does this imply that the alignment-based method is much better than the joint method? It's not reasonable since the clustering performance of alignment-based methods may not necessarily be better than that of joint methods.

3. Is the complexity of computing Shapley values truly O(n!)? When dealing with a larger number of views, can this evaluation framework still be utilized for computation?

4. Are the loss functions L in Eqs (15) and (16) on page 6 the same? If so, there is a problem of inconsistent dependent variables. In addition, $D_ij$ in Eq. (9) is a scalar and should not be bolded.
questions: questions: The alignment-based method proposed in Theorem 1 will make the contribution values of several views the same. Combined with the experimental results in Table 2, does this mean that the end point of the view contribution optimization proposed in this paper is contrastive learning? If not, please explain in detail the difference between the method in this paper and the contrastive learning method?
limitations: limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work.
paper_summary: paper_summary: This paper studies multi-view clustering and seeks to investigate the view cooperation issue. The authors consider DMVC as an unsupervised cooperative game and regard each view as a participant. Compared with the existing methods, this consideration is new and interesting. Based on the novel idea, the authors proposed SCE-MVC, a novel shapley-based cooperation enhancing multi-view clustering method. The paper is well-organized. The experiments are convincing.
strength_weakness: strength_weakness: 1. Figure 2 is confusing. The specific structure of View Cooperation Enhancing Module is not clearly presented.

2. There are many formulas and symbols. It is suggested to add a notation table.

3. Although the authors try to explain model (1), it is still difficult to understand Shapley Value from the model. In addition, many variables are not clearly explained. The authors should present more information about the model and explain all variables used in this model, such as S_i, {i}, s\{i}， etc.
questions: questions: The article involves rich theoretical and mathematical knowledge. I have a question to the designation of the model: Which design is the key to improving model performance?
limitations: limitations: NA

Comments: title: Thanks for your detailed responses.
Thanks to the reply.
Follow Up for reviewer CDST
Follow Up for reviewer TWtF
Follow Up for reviewer v5TM
Official Review of Submission781 by Reviewer v5TM
comment: I have carefully reviewed the authors' responses. Since all my concerns have been tackled, I understood the motivation, contribution, and experimental analysis more clearly. Combined with the comments from other reviewers, I decided to raise the score of this paper.
Thanks to your kindly responses. Most of my concerns have been resolved.
Thank you for your valuable insights on our work. We will eloquently present our idea of view cooperation in the final version.
Thank you for your positive comments. Cooperation among views offers a fresh perspective for the DMVC methods.
Thank you for acknowledging the motivation of our work. The dynamics and cooperation between views is worthy of study.
Thanks for your responses, my questions have been resolved. Additionally, I checked out the feedback of other reviewers. I think the motivation of this paper commendable. I decide to elevate the rating of the paper to weak accept.
Thank you for your responses, which have addressed my concerns. I am favor of the coopearation idea for MVC and vote for acceptance.
Thanks for the response. All my concerns have been addressed. After reading all the reviewers' comments and responses, I decided to raise the score.
Dear Reviewers, Area Chairs, Senior Area Chairs and Program Chairs,

We sincerely thank the efforts and constructive comments you have made for this paper. The reviewers put forward many insightful questions and valuable suggestions towards improving our paper. In the rebuttal phase, we provided detailed responses to all reviewers' comments point by point, hoping to address the issues raised by reviewers. The discussion period is coming to an end, and we are actively awaiting for further discussion from Reviewers. If you have any other questions, we are happy to discuss them with you at any time. If our response addresses your concerns, please consider increasing your rating. We are looking forward to your reply.

Best,
Authors of paper-781.