View Gap Matters: Cross-view Topology and Information Decoupling for Multi-view Clustering

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-24 13:34:04

Abstract: Multi-view clustering, a pivotal technology in multimedia research, aims to leverage complementary information from diverse perspectives to enhance clustering performance. The current multi-view clustering methods normally enforce the reduction of distances between any pair of views, overlooking the heterogeneity between views, thereby sacrificing the diverse and valuable insights inherent in multi-view data. In this paper, we propose a Tree-Based View-Gap Maintaining Multi-View Clustering (TGM-MVC) method. Our approach introduces a novel conceptualization of multiple views as a graph structure. In this structure, each view corresponds to a node, with the view gap, calculated by the cosine distance between views, acting as the edge. Through graph pruning, we derive the minimum spanning tree of the views, reflecting the neighbouring relationships among them. Specifically, we applied a share-specific learning framework, and generate view trees for both view-shared and view-specific information. Concerning shared information, we only narrow the distance between adjacent views, while for specific information, we maintain the view gap between neighboring views. Theoretical analysis highlights the risks of eliminating the view gap, and comprehensive experiments validate the efficacy of our proposed TGM-MVC method.

Main paper: 
 related work 2.1 deep multi-view representation learning: 
 Recently, Multi-View Clustering (MVC) [1, 20-26, 34, 37, 38, 51] has recently attracted considerable attention as a crucial multimedia technology. Within this domain, deep Multi-View Clustering (DMVC) [2, 8, 18, 19, 30, 32, 42, 46, 53] using deep networks has emerged as an important approach. Presently, deep multi-view clustering methods can be categorized into three main classes: joint methods, alignment-based methods, and share-specific methods.Joint methods consider the differences and complementarities between views, achieving the representations by independently optimizing and concatenating sample representations in each view space. For instance, DMJC [41] involves training independent autoencoders for each view and then utilizes sharpening of the distribution of concatenated representations as a self-supervised signal for training. While joint method for multi-view representation learning is straightforward and effective, it lacks direct interaction between views, thereby hindering the acquisition of a consensus across multiple views. Alignment-based methods, on the other hand, map representations from different views to a shared semantic space based on the consistency of multi-view data, with the most typical approach being contrastive learning to minimize the distances between any two views. For instance, MFLVC [47] introduces two objectives on high-level features and pseudo-labels, leveraging contrastive learning to diminish the distances between views and achieve multi-view clustering. However, this method primarily focuses on the consensus among views, neglecting the differences and gaps between views, leading to the loss of unique view-specific information during the alignment process and resulting in information loss. Share-specific methods offer a more comprehensive integration of the aforementioned approaches by decoupling representations into consensus information and unique information, thus considering both the consistency and complementarity of multiple views. Nevertheless, the share-specific information architecture still exhibits shortcomings in addressing the view gap. In the subsequent section, we will delve into a detailed illustration of the view gap issue.Recent studies have unveiled the existence of gaps among heterogeneous data sources, and forcefully eradicating these distinctions could detrimentally affect the data's representation learning. Wang et al. [36] carried out a theoretical scrutiny on contrastive loss, stressing that enhanced alignment should involve the diminishment of disparities across diverse modalities. Nonetheless, although alignment is extensively applied in pre-training utilizing multiple data sources, potential conflicts could emerge between upstream alignment objectives and diverse downstream tasks, such as classification or clustering. Jiang et al. [11] analyzed two modalities, image and text, and discovered that minimizing the modality gaps  
method: 
 In this section, we elaborate on our Tree-Based View-Gap Maintaining Multi-View Clustering (TGM-MVC) method. The entire framework is illustrated in Fig. 2 . It encompasses three modules and three stages: 1) In the first stage, we employ a conventional share-specific Learning Module (ShaSpec) to decouple sample features X (𝑣) into shared representations Z (𝑣) and specific representations Ẑ(𝑣) . 2) In the second stage, we introduce a Shared Tree-based View Consensus Learning Module (ShaTree) and a Specific Tree-based View Gap Maintaining Module (SpecTree), which respectively operate on the shared representations Z (𝑣) and specific representations Ẑ(𝑣) obtained from the first module. 3) In the third stage, we concatenate shared and specific representations from all views for clustering. For clarity, all symbols and their meanings are presented in Table 1 .Given a set of multiview data X = {X (1) , X (2) , ..., X (𝑉 ) }, where 𝑉 is the number of views.X (𝑣) = {x (𝑣) 1 , x (𝑣) 2 , ..., x (𝑣) 𝑁 } ∈ R 𝑁 ×𝑑 𝑣represents the sample set of the 𝑣-th view, with 𝑁 as the number of samples, and 𝑑 𝑣 as the input dimension of the 𝑣-th view. x (𝑣) 𝑖 denotes the 𝑖-th sample of the 𝑣-th view.We extract sample x (𝑣) 𝑖 into two types of features, that is the shared view feature z (𝑣) 𝑖 and the specific view feature ẑ(𝑣) 𝑖 , in the manner of [10, 27, 29, 44] . To achieve this, we trained specific encoders Ê(𝑣) and specific decoders D(𝑣) tailored to each view, along with a universal shared encoder E and a shared decoder D for all views. It should be noted that in E or D, all components are the same across all views except for the first and last two linear mapping layers. The ultimate objective is to achieve the optimal clustering performance by concatenating the shared and specific representations from individual views.To exploit the consensus and heterogeneity between views, we leverage the share-specific representation learning module (ShaSpec) in the Stage 1 to decouple the sample features into shared representations and specific representations. The loss within the ShaSpec module is composed of three components: reconstruction loss 𝐿 𝑟𝑒𝑐 , contrastive learning loss 𝐿 𝑐𝑜𝑛 , and orthogonal loss 𝐿 𝑜𝑡ℎ . We begin by decoupling the representations of the samples, withEQUATIONfor 𝑖 ∈ {1, ..., 𝑁 } and 𝑣 ∈ {1, ..., 𝑉 }. To prevent feature collapse, we trained a corresponding decoder for each encoder to reconstruct the sample features, withEQUATIONwhere X (𝑣) and X(𝑣) respectively represent the reconstruction of shared representation Z (𝑣) and specific representation Ẑ(𝑣) on the 𝑣-th view. Then we optimize the reconstruction loss of all views byEQUATIONwhere the reconstruction loss of shared and specific representations from all views are aggregated. Shared Representations Learning: The ShaSpec module focuses on the learning of shared representations { Z (𝑣) } 𝑉 𝑣=1 using contrastive learning to gather consensus information among views. Its objective is to minimize the differences between shared representations from different views, thereby bringing them closer together. The contrastive loss ℓ 𝑖𝑛𝑠 (𝑝, 𝑞) between the 𝑝-th view and the 𝑞-th view is represented as:EQUATIONEQUATIONwhere ∫ (•, •) is the cosine similarity and 𝜏 𝑙 is the temperature hypothesis. Then the overall contrastive loss L 𝑐𝑜𝑛 between any two views is given by: EQUATION𝐸 (1) 𝐷 ( 1) 𝐗 (𝟏) 𝐗 (𝟒) 𝐸 ( 4) 𝐷 ( 4) 𝐗 (𝟒) … … 𝐗 (𝟏) 𝐗 (𝟒) ෨ 𝐸 … ෩ 𝐷 … … View Heterogeneity Distance Matrix 𝐿 𝑟𝑒𝑐 𝐿 𝑟𝑒𝑐 𝐿 𝑟𝑒𝑐 View Consensus Distance Matrix 𝐿 𝑜𝑡ℎ ෩ 𝐗 (𝟏) ෩ 𝐗 (𝟒) 𝐙 (𝟏) 𝐙 (𝟐) 𝐙 (𝟑) 𝐙 (𝟒) ෨ 𝐙 (𝟏) ෨ 𝐙 (𝟑) ෨ 𝐙 (𝟐) ෨ 𝐙 (𝟒) ෨ 𝐿 ෨ 𝐙 (𝟏) ෨ 𝐙 (𝟐) ෨ 𝐙 (𝟑) ෨ 𝐙 (𝟒) 𝐿 𝐙 (𝟏) 𝐙 (𝟐) 𝐙 (𝟑) 𝐙 (𝟒) (a)EQUATIONwhere ∫ (•, •) represents the cosine similarity between sample features.In Stage 1, we train the ShaSpec module for 𝑇 1 rounds to decouple samples into view consensus information and view heterogeneity information, enabling a detailed study of the relationships between views in the subsequent stage. The overall loss of Stage 1 learning consists of the reconstruction loss L 𝑟𝑒𝑐 , the contrastive loss L 𝑐𝑜𝑛 and the orthogonal loss L 𝑜𝑡ℎ , i.e.,EQUATIONwhere L 𝑜𝑡ℎ and L 𝑐𝑜𝑛 with 𝜆 1 and 𝜆 2 are the weights balancing the three loss terms.During the Stage 1, we aim to align the consensus representations of all views using a contrastive learning paradigm to achieve uniform representations across all views. However, based on our assumption, there exists view gaps between different views. Disregarding the view gap and forcibly aligning the representations of arbitrary pairs of views would lead to the loss of rich multi-view information. Therefore, we seek to rectify the consensus representations obtained from Stage 1 through the Shared Tree-based View Consensus Learning Module (ShaTree), obtaining consensus from adjacent view pairs instead of all views.To quantify the proximity relationships between views, we construct the View Consensus Distance Matrix M ∈ R 𝑉 ×𝑉 , where M 𝑝𝑞 represents the cosine similarity distance of the shared representations between the 𝑝-th view and the 𝑞-th view:EQUATIONwhere the distance between views is defined as the average cosine similarity distance between corresponding samples of the two views.If we we conceptualize multiple views as a graph structure, considering each view as a node and the distance between views as the edge length between nodes, then the multi-view can be modeled as a view graph G = (X, E), which contains 𝑉 nodes X = {X (1) , X (2) , ..., X (𝑉 ) } and 𝐶 2 𝑉 edges E = {(X (𝑝 ) , X (𝑞) ) | 𝑝, 𝑞 ∈ [1, 𝑉 ] , 𝑝 < 𝑞}. To simplify the representation, we denote the edge (X (𝑝 ) , X (𝑞) ) as (𝑝, 𝑞).To discover the neighborhood relationships between consensus information of views, we employ the Prim algorithm [5, 7] on the View Consensus Distance Matrix M to generate a minimum spanning tree, obtaining a subgraph G = (X, E) of G, which edge set E = {(𝑝 1 , 𝑞 1 ), (𝑝 2 , 𝑞 2 ), ..., (𝑝 𝑉 -1 , 𝑞 𝑉 -1 ))}.To guarantee diversity in view information and avoid the direct merging of shared representations from distant views, we exclusively bring closer the shared representations among view nodes that correspond to the edges encompassed in the View Consensus Spanning Tree G. Hence, we adjust Eq.( 6) in the Stage 1 as:EQUATIONwhere ℓ 𝑖𝑛𝑠 (𝑝, 𝑞) is the contrastive loss between two views, i.e., X (𝑝 ) and X (𝑞) , as computed in Eq.( 5).The specific representation Ẑ(𝑣) learned in Stage 1 is primarily constrained by the orthogonal loss and the reconstruction loss. There remains a risk that the specific representations of various views lack the necessary distinctiveness since the above two constraints can hardly ensure that the learned Ẑ(𝑣) fully captures the heterogeneity inherent in different views. Given the insights from pertinent information theory [3] , we introduce the concept of information gap 𝛿 𝑝𝑞 to highlight the potential consequences of homogenizing the representations across different views for downstream tasks. Such 𝛿 𝑝𝑞 indicates the difference in the amount of information provided by the two views for the clustering task. We introduce the theoretical optimal output Y * in the solution space, such that the information gap 𝛿 𝑝𝑞 can be defined as:EQUATIONwhereY * = arg max Y *𝑁 𝑀𝐼 (Y * , Y). 𝐼 (X (𝑣) ; Y * ) illustrates the amount of information that the 𝑣-th view can contribute to the clustering task. It is noteworthy that Y is a clustering label used for theoretical elucidation, thus our task remains inherently unsupervised.Theorem 1. Suppose there exists encoders ℎ 𝑝 : X (𝑝 ) → Z (𝑝 ) and ℎ 𝑞 : X (𝑞) → Z (𝑞) , such that Z (𝑝 ) = Z (𝑞) . And the fusion functions 𝑔 𝑥 and 𝑔 𝑧 , applying to {𝑋 (𝑣) } 𝑣=𝑝,𝑞 and {𝑍 (𝑣) } 𝑣=𝑝,𝑞 respectively, allow the fused features to retain maximum information content. Then encoders ℎ 𝑝 and ℎ 𝑞 would disregard the view gap, resulting in information loss:EQUATIONTheorem 1 elucidates that as the representations of two views converge completely, it leads to information loss and impairs the performance of downstream tasks. Additional theoretical analysis can be found in Appendix A.According to [36] , contrastive learning continuously narrows the gap between the two views when a plentiful amount of negative samples are available. To preserve the view gap, apart from avoiding a direct convergence of shared representations Z (𝑣) between distant views, we also impose constraints on the view-specific features using a similar approach. Specifically, we generate the View Heterogeneity Distance Matrix M ∈ R 𝑉 ×𝑉 based on the view-specific representations Ẑ(𝑣) obtained in the Stage 1:EQUATIONwhere ∫ (ẑ(𝑝 ) 𝑖 , ẑ(𝑞) 𝑖 )is the cosine similarity between the two representations.Similar to the View Consensus Spanning Tree G, we construct a View Heterogeneity Spanning Tree Ĝ = (X, Ê) based on the View Heterogeneity Distance Matrix M using the Prim algorithm, where the pair of views (𝑝, 𝑞) in Ê indicates a higher similarity in specific representations for the 𝑝-th view and the 𝑞-th view. To maintain the view gap between these two views, we design the loss as:EQUATION)EQUATION)where l𝜇= 1 2𝑁 𝑁 𝑖=1 l (𝑝 ) 𝑖 + l (𝑞). Hence, the purpose of ℓ 𝑠𝑝 (𝑝, 𝑞) is to minimize the variance of the contrastive losses incurred by any sample in the 𝑝-th view and the 𝑞-th view.Theorem 2. Imposing the constraints of ℓ 𝑠𝑝 (𝑝, 𝑞) can prevent the convergence of representations from two views into uniformity, i.e. Ẑ(𝑝) ≠ Ẑ(𝑞) , thereby preserving the diverse information present across the multiple views.The proof of Theorem 2 is relocated to the Appendix B. In regard to view pairs within View Heterogeneity Spanning Tree Ĝ, we impose constraints using Eq.( 15), thereby crafting the loss for specific representations as:EQUATION)In Stage 1, we solely employ the ShaSpec module for training, decoupling the representations into view-shared and view-specific representations using Eq.( 8).Input: The multi-view raw features {X (𝑣) } 𝑉 𝑣=1 ; the interation number 𝑇 1 and 𝑇 2 .Output: The clustering result R.1: for 𝑖 = 1 to 𝑇 1 do 2:Obtain the shared representations { Z (𝑣) } 𝑉 𝑣=1 and the specific representations { Ẑ(𝑣) } 𝑉 𝑣=1 by ShaSpec Module using Eq.( 8). 3: end for 4: while the total loss hasn't converged do 5:Calculate View Consensus Distance Matrix M and View Heterogeneity Distance Matrix M using Eq.( 9)(13).Obtain View Consensus Spanning Tree G = ( X, E ) and View Heterogeneity Spanning Tree Ĝ = ( X, Ê ) using the Prim algorithm.for 𝑗 = 1 to 𝑇 2 do 8:Calculate the total loss 𝐿 𝑡𝑜𝑡𝑎𝑙 using Eq.( 17).Updating the network with Adam Optimizer by minimizing 𝐿 𝑡𝑜𝑡𝑎𝑙 .end for 11: end while 12: Concatenate shared and specific representations across all views using Eq.( 18) and then clustering. 13: return R To uphold the view gaps between views and ensure the richness of information across multiple views, we calibrate Stage 1 through Stage 2. Every 𝑇 2 rounds, we reassess the View Consensus Spanning Tree and View Heterogeneity Spanning Tree, optimizing the entirety through reconstruction loss, orthogonality loss, and two-part losses acting upon shared and specific representations:EQUATIONwhere 𝜆 1 , 𝜆 2 and𝜆 3 are the weights for different parts of loss respectively. Meanwhile, the contrastive lossL 𝑐𝑜𝑛 in Stage 1 is replaced by L in Stage 2.Upon completion of training in Stage 2, we concatenate the shared and specific representations across all views to derive the holistic representation Z of the samples, which is subsequently employed for the final clustering task:EQUATIONwhere symbol and ⊕ both denote the concatenation on the feature dimension. The detailed learning process of our proposed TGM-MVC is shown in Algorithm 1. 
experiments 4.1 datasets: 
 To demonstrate the efficacy of our TGM-MVC approach, we conduct elaborate experiments on six benchmark datasets: Synthetic3d, Cora, ReutersEN, Caltech101, ALOI-100, and STL10. The fundamental characteristics of these six datasets are illustrated in Table 2 . In this section, we validate the effectiveness of our proposed TGM-MVC by addressing the following five inquiries:• Q1: How effective does the TGM-MVC method exhibit in the realm of deep multi-view clustering tasks? • Q2: How does the View Consensus Spanning Tree influence the performance of TGM-MVC? • Q3: How does the View Heterogeneity Spanning Tree influence the performance of TGM-MVC? • Q4: How do the hyper-parameters impact the performance of TGM-MVC? • Q5: Does the TGM-MVC model exhibit convergence?The experimental setup includes an Intel Core i7-7820x CPU, NVIDIA GeForce RTX 3090 GPU, and 64GB of RAM. For software support, the experiments were carried out using the PyCharm platform. Additionally, the training process made use of the Adam optimizer.We conducted a comparative analysis between the TGM-MVC method and eight state-of-the-art multiview clustering algorithms: DEMVC [43] , FMCNOF [49] , CoMVC [32] , SiMVC [32] , SDMVC [45] , MFLVC [47] , DSMVC [31] , and SFMC [14] .In order to assess the efficacy and superiority of our TGM-MVC approach, we utilize commonly employed metrics, namely clustering accuracy (ACC), normalized mutual information (NMI), and purity (PUR) [33, 35, 54] .To validate the efficacy of our proposed method, we conducted comparisons with eight benchmark methods across six datasets. The clustering results of our model and other algorithms are presented in Table 3 . From Table 3 , the following conclusions can be drawn:(1) Our approach outperformed or matched existing state-of-theart methods in three metrics across the six datasets. With respect to Purity, our method outshines the baseline algorithms, especially on the Caltech101 and ALOI-100 dataset, which surpasses the secondbest algorithm by 8.55% and 6.22%.(2) These datasets cover multiple views (ranging from 3 to 5 views), and our method excelled on most of these datasets, whereas other methods achieved satisfactory clustering results on only one or some of the datasets. We attribute this to our methodology's acknowledgment of the view gaps, thereby demonstrating strong performance on datasets with more views.To verify the effectiveness of the ShaTree module and SpecTree module, we conducted ablation experiments on six datasets. Specifically, we held constant the Share-Specific Representation Learning ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ALOI-100 without ShaTree ALOI-100 with ShaTree (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ෩ 𝒁 (1) ෩ 𝒁 (2) ෩ 𝒁 (3) ෩ 𝒁 (4) ALOI-100 without ShaTree ALOI-100 with ShaTree (3) The ShaTree module and SpecTree module synergize effectively, with the combined utilization of these two modules resulting in a greater improvement in clustering performance. Anonymous Authors 𝒁 (1) 𝒁 (2) 𝒁 (3) 𝒁 (4) 𝒁 (1) 𝒁 (2) 𝒁 (3) 𝒁 (4) 𝒁 ( 1) 𝒁 ( 2) 𝒁 ( 3) 𝒁 ( 4) 𝒁 (1) 𝒁 (2) 𝒁 (3) 𝒁 (4) ALOI-100 without SpecTree ALOI-100 with SpecTree 𝒁 (𝟐) 𝒁 (𝟑) 𝒁 (𝟒) (b) Fully-connected view graph without SpecTree 𝒁 (1) 𝒁 ( 2) 𝒁 ( 3) 𝒁 ( 4) 𝒁 (1) 𝒁 (2) 𝒁 (3) 𝒁 (4) 𝒁 ( 1) 𝒁 ( 2) 𝒁 ( 3) 𝒁 ( 4) 𝒁 (1) 𝒁 (2) 𝒁 (3) 𝒁 (4) ALOI-100 without SpecTree ALOI-100 with SpecTree In actual experiments, we kept the temperature hyperparameter 𝜏 𝑙 for contrastive learning fixed at 0.1 and the weight 𝜆 1 for orthogonal loss L 𝑜𝑡ℎ fixed at 1, adjusting only the weight 𝜆 2 for shared representation loss L and the weight 𝜆 3 for specific representation loss L.In order to assess the robustness of our model, we conducted a sensitivity analysis on 𝜆 2 and 𝜆 3 . From the results in Fig. 5 , we discern a notable influence of 𝜆 2 on the model's performance, while the impact of 𝜆 3 is relatively minor. This is because 𝜆 2 is also involved in the training of the first stage, hence changes in 𝜆 2 have a stronger impact on the overall training effectiveness of the model.We completed experiments to demonstrate the convergence of the model on six datasets. Due to space constraints, we only demonstrate the changes in ACC and NMI with the number of training epochs on the ALOI-100 and Cora datasets, as shown in Fig 6 . Experiments prove the convergence of our proposed TGM-MVC.  
conclusions: 
 In this paper, we introduce a novel approach for modeling multiple views utilizing a innovative spanning tree topology. Our proposed TGM-MVC method aims to preserve the distinctiveness among views, thereby enhancing the efficacy of multi-view clustering. Specifically, we separately generate minimum spanning trees for shared representations and specific representations, relaxing the constraints between non-adjacent views in shared representations while maintaining the distances between adjacent views in specific representations. We provide theoretical justification for the importance of maintaining view gaps and demonstrate the effectiveness of our approach through comprehensive experiments.  
: 
 Multi-view learning is a nontrivial topic in the filed of multimedia technology. A fundamental issue of multi-view learning is multi-view representation learning [6, 12, 15, 17, 28, 32, 44] , which addresses the challenge of unifying data representation from diverse sources. Multi-view clustering (MVC) [3, 16, 18, 40, 45, 48, 50, 52 ] is a typical task in multi-view learning, where the clustering performance is largely contingent upon the quality of the representations of multi-view samples. As such, exploring representation learning across multiple views holds significant importance, especially within the domain of multi-view clustering tasks.Existing deep multi-view clustering representation learning methods can be categorized into three paradigms [9, 15] , i.e., joint methods [13, 39, 41, 55] , alignment-based methods [6, 12, 32, 52] , and share-specific methods [10, 27, 29, 44] . Joint methods enable samples from different views to be independently optimized within their respective view spaces, and then integrates the representations from different views through fusion techniques. The latter two methods both map representations from different views into a shared subspace, aiming to learn the unified representation for all views. Alignment-based methods [6, 12, 32, 52] aim to bring representations closer between any view pair. Share-specific methods [10, 27, 29, 44] model the view information as shared information and specific information, maximizing the mutual information of shared information across any view.Alignment-based methods and share-specific methods have demonstrated notable advancements in the MVC task. However, their capacity to fully apprehend the intricacies of view heterogeneity appears to be constrained. Alignment-based methods concentrate solely on the consensus between views, overlooking the complementary information among them. Share-specific methods forcibly bring views closer in consensus, and lack supervision in learning complementary information as well. In real-world scenarios, views exhibit intricate adjacency relationships. Fig. 1 presents t-SNE visualizations of different views from ALOI-100 dataset. In Fig. 1 (a)(b)(c), we utilized alignment-based method, share-specific method, and our proposed TGM-MVC method for the same training epochs. The contrastive-based method homogenizes different views, leading to subpar clustering performance; whereas the TGM-MVC method demonstrates excellent clustering performance, maintaining the distinctions between views (Fig. 1(d) ). In this paper, we define the views' distinctions as view gap, which is measured by cosine distance of view representations. Based on the concept of view gap, we propose a foundational hypothesis, i.e. the pursuit of uniform consensus across all views may be inherently irrational. Instead, preserving the view gaps between distinct views and enriching their diversity should facility the learning of more robust multi-view representations.In line with the limitations highlighted above for current sharespecific methods, we introduce a novel approach, termed the Tree-Based View Coordination Enhancement (TGM). Building upon the share-specific framework, our proposed method incorporates a more comprehensive understanding of the relationships between multi-views. Our core idea is the recognition of the intricate adjacency relationships that exist between views in real-world scenarios, as demonstrated by the t-SNE visualizations in Fig. 1 (a) and (b). Unlike alignment-based methods that focus merely on achieving consensus or share-specific methods that force views into closer alignment, our TGM method takes into account both consensus and complementary information. To achieve this, we begin by constructing a view adjacency matrix for the shared representations, capturing the inherent relationships across different views. With this matrix, we generate a minimum spanning tree that encapsulates the shared information while preserving the natural distances between views. By bringing closer the shared representations between adjacent nodes in this view tree, we avoid the limitations of directly aligning distant views, as observed in Fig. 1(d) .The primary contributions of this work can be summarized as follows:• Our TGM-MVC method acknowledges the intrinsic "view gap" between views. Through theoretical and empirical research, we demonstrate that maintaining this view gap contributes to preserving the rich information of multi-views. • The tree-based view graph generation strategy allows for the rapid and efficient construction of neighborhood relationships among views, analyzing the proximity relationships between views, thus facilitating downstream tasks in multiview learning.• Comprehensive experiments across six benchmark datasets serves to highlight both the superiority and efficiency of the proposed TGM-MVC method. Furthermore, the effectiveness of our approach is substantiated by ablation studies and visualization experiments. 
 
 Appendix:  


Decision: This paper considers the diverse and insightful inherent in multi-view data to build a tree-based view gap maintaining multi-view clustering algorithm. They conceptualize multiple views as graph structures, followed by graph pruning, which is similar to tree decision schemes. The given share-specific learning framework generates view trees for both shared and view-specific information. The authors also offered a theoretical analysis of the risks of eliminating the view gaps. Experimental results show its efficacy of the proposed method.

We have five reviewers and all of them voted for accepting this paper, due to good presentation, well organization of the whole paper, general well-designed algorithm, and sufficient experiments on multi-view learning. After rebuttal, all the reviewers are satisfied and this paper is good to be presented at ACM MM 2024.

Reviews: review: paper_summary: paper_summary: This paper proposes a novel multi-view clustering method called Tree-Based ViewGap Maintaining Multi-View Clustering (TGM-MVC). TGM-MVC considers multiple views as a graph structure, in which each view is a node, with the view gap is the edge. Then, the minimum spanning tree of the views can be derived by graph pruning, which can reflect the neighbouring relationships among views. For view-shared information, the distance between adjacent views should be narrowed. Meanwhile, for view-specific information, the distance between adjacent views should be maintained. Theoretical analysis and comprehensive experiments can show the effectiveness of the proposed TGM-MVC.
strength_weakness: strength_weakness: 1.​The tree-based view learning is interesting and novelty, which can be readily implemented in deep multi-view learning framework.
2.​The framework is lucidly expressed and the paper is easy to follow.
3. The paper is well-organized and clearly written.
limitations: limitations: 1. It would be interesting to discuss what are the essential differences or benefits of tree-based view learning I think it’s interesting, but more details about it should be discussed.
2. The datasets in experiments looks some old, and more compasions should be added.
paper_summary: paper_summary: The author introduces the concept of view gap and acquires the adjacency relationships between views by generating a minimum spanning tree of the view graph. The shared and specific representations of samples are learned separately, ultimately achieving favorable clustering results.
strength_weakness: strength_weakness: 1. The author proposes a novel definition of view graph. By considering views as nodes and the distances between views as edges, multi-view model is transformed into a graph structure. Then, by generating a minimum spanning tree, the adjacency relationships between views are represented using a tree structure.
2. The author proposes the perspective of maintaining the view gap, conducts theoretical analysis, and visualizes it in experiments. Additionally, through experimental results, the significance of maintaining the view gap for the Multi-View Clustering (MVC) task is demonstrated.
3. The proposed method works well on six datasets which contain different number of views, achieving significant performance gains compared to existing MVC competitors.
limitations: limitations: 1. Learning the shared representations using the ShaSpec Module not only brings the neighboring views closer but also tends to bring the shared representations of all views closer together. Given this broader impact, why the approach isn't simply to bring together the shared representations across all views directly. Could you elaborate on the advantages of specifically employing the ShaSpec Module in this context?
2. MVC is inherently an unsupervised task, yet in the depiction of the information gap (Equation 11), Y* is derived from the label Y. Given this context, is it justifiable to use the label Y when defining the information gap?
3. The paper notes that the process of regenerating the minimum spanning tree within the model is iterative. Could you clarify how many iteration epochs are involved in this process?
4. The reference for the information gap (line 501) is "A novel approach for effective multi-view clustering with information-theoretic perspective.", which was made in error by mistake. The correct citation should be "Understanding and constructing latent modality structures in multi-modal representation learning."
paper_summary: paper_summary: This paper proposes a new method called TGM-MVC (Tree Based View Map Maintaining Multi View Clustering) for solving multi view clustering problems. Traditional multi view clustering methods typically attempt to reduce the distance between any two views, but overlook the heterogeneity between views, resulting in the loss of valuable information in multi view data. TGM-MVC constructs a graph structure between views, where each view is a node and the gaps between views (calculated by cosine distance) are used as edges. By pruning the graph, the minimum spanning tree of the view is obtained, reflecting the adjacency relationship between them.
strength_weakness: strength_weakness: The paper proposes a tree based view gap preserving multi view clustering (TGM-MVC) method, which views multiple views as graph structures, with each view corresponding to a node and the gaps between views as edges, allowing for modeling of heterogeneity between views. By pruning the graph, the minimum spanning tree of the view is obtained, reflecting the adjacency relationship between views, which helps to identify and preserve complementary information between different views.Adopting a shared specific learning framework, which only reduces the distance between adjacent views for shared information and maintains view gaps for specific information, is beneficial for maintaining view diversity.
limitations: limitations: The reference format is inconsistent, with some references having their year placed after the author's name, while others are placed at the end of the paper title.
paper_summary: paper_summary: This paper proposes a Tree-Based Gap-Maintaining Multi-View Clustering (TGM-MVC) method. The approach introduces a novel concept of conceptualizing multiple views as a graph structure, where each view corresponds to a node and the differences between views as the edges. The minimum spanning tree is generated to reflect the neighboring relationships between views. Both theoretical analysis and comprehensive experiments demonstrate the effectiveness of our proposed method.
strength_weakness: strength_weakness: Novelty:I think that the perspective of maintaining the view gap between views presented in this paper is novel.

Theoretical Foundation:The theoretical foundation of the paper is robust, introducing the concept of the information gap to illustrate the problems with completely bringing different views closer together, thereby providing support for the perspective of maintaining the view gap.

Experiment Results:The experiments in the paper are thorough, utilizing topological graph visualizations of the view gap to demonstrate the effectiveness of the proposed ShaTree and SpecTree modules.
limitations: limitations: Visualization: In the visualization of the view gap, as depicted in Figure 1, it is intriguing to consider whether similar view gaps might be present in other datasets. To explore this further, it would be beneficial to conduct a comparative analysis using an additional dataset. Please provide another dataset to demonstrate whether this view gap phenomenon is consistent across different data contexts?

Experiments: In the ablation experiment (Section 4.4) results in Table 4, why are the results of using both ShaSpec and ShaTree modules worse than using only ShaSpec module? Does this mean that ShaTree has a negative effect on clustering? Please provide a detailed discussion of this phenomenon. The TCE-MVC method has three hyperparameters (lambda1, lambda2, lambda3). However, in the Hyper-parameter Analysis (Section 4.5), only lambda2 and lambda3 were analyzed. Why wasn't lambda1 analyzed? Please provide a sensitivity analysis for lambda1.

Citation:The six datasets used in the experiments lack citations, which I believe is not in accordance with standard practices.
paper_summary: paper_summary: This manuscript proposes a Tree-Based View Gap Maintaining Multi-View Clustering (TGM-MVC) method. TGM-MVC reflects the affinity relationship between views by pruning the view graph, which provides a new insight for the MVC task. In addition, the sufficient experiments can verify the effectiveness of the approach.
strength_weakness: strength_weakness: Although there have been some MVC methods based on the decoupling of specific and common representations before, this manuscript is more suitable for processing the two representations based on the minimum spanning tree of the view. Therefore, I think this article is quite innovative. The experiments in this paper are comprehensive, which fully verify the effectiveness and superiority of the proposed method. This manuscript is well organized and clear for others to follow.
limitations: limitations: Regarding the understanding of view graph, I have a different opinion from the authors. First of all, I think that the view graph at the view level cannot measure the relationship between different views of all samples, because the proximity relationships between views are different at the fine-grained sample level. In addition, although the proposed method maintains the view gap, the most essential cluster structure of the cluster cannot be seen in the results in Figure 1.

In Stage 3 of Figure 2, it is depicted that all shared and specific representations are concatenated. However, the method used to derive the final clustering result from this amalgamation is not clearly explained. Please provide more details on how the final clustering outcome is obtained?

The placement of Q1 to Q5 proposed in section 4.1 is unreasonable, which should be placed between heading 4 and subheading 4.1.

The paper lacks an introduction to the comparative algorithms, among these algorithms I noticed that the SFMC algorithm exhibits significant performance differences across various datasets. Why does the performance of SOTA method SFMC drops dramatically on the STL 10 and ReutersEn datasets?