Heterogeneity-Aware Federated Deep Multi-View Clustering towards Diverse Feature Representations

Authors: xiaorui  jiang
zhongyi  ma
yulin  fu
yong  liao
yuan  peng
  zhou
peng  yuan
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 13:36:27

Abstract: Multi-view clustering has garnered increasing attention in recent years because of its ability to extract consistent and complementary information from multi-view data. In this context, contrastive learning has often been employed to explore the common semantics across different views. However, we notice that existing multi-view contrastive clustering methods often overlook cases where samples belonging to the same cluster but different views are incorrectly classified as negative feature pairs, leading to larger separations between features belonging to the same cluster in the feature space. To address this issue, we propose to shift the perspective from the view-level to the cluster-level and introduce $\mathbf{C}$luster-level $\mathbf{C}$ontrastive $\mathbf{D}$eep $\mathbf{M}$ulti-$\mathbf{V}$iew $\mathbf{C}$lustering ($\mathbf{CCDMVC}$) method based on an intra-cluster negative pair exemption strategy. Specifically, by constructing global features to utilize complete view information, we infer the clustering probability of each sample, thus reducing the construction of negative feature pairs belonging to the same cluster. As a result, the contrastive loss is corrected, allowing the model to treat different levels of feature pairs differently, minimizing the introduction of noise and making the sample points within the same cluster more compact. Additionally, we propose a cluster-level imputation module to make CCDMVC compatible with scenarios involving incomplete data. This module infers missing features with high confidence clustering probabilities and classifies them in cluster-level form. We conduct extensive experiments on eight datasets with fourteen baseline algorithms. The results demonstrate that CCDMVC exhibits superior clustering performance.

Main paper: 
 introduction: 
 Perceiving the same thing from multiple perspectives often allows for a more comprehensive understanding of the information. In recent years, the rapid development of multimedia technology [10, 12] has generated a substantial amount of unlabeled multimodal data, posing a challenge for clustering analysis of multi-view data. In response, Multi-View Clustering (MVC) [2, 3, 5, 6, 11] methods have been proposed and gained momentum. These methods expedite the exploration of consistent and complementary information within the data by fostering collaboration across multiple views, thereby achieving a comprehensive clustering structure.However, most existing MVC methods [23, 41, 44, 46] are built on the assumption of centralization, presuming that multi-view data is stored within a single entity, which proves challenging to apply in many real-world scenarios. For example, patients may undergo relevant examinations at different hospitals. Only through the aggregation of the medical data can their health status be accurately reflected. However, the simplistic consolidation of this data may give rise to privacy concerns [25] . To address this challenge, Federated Learning (FL) [17, 35] presents a promising approach by aggregating individual client models through a central server while preserving the privacy of raw data. The integration of FL with MVC, referred to as Federated Multi-View Clustering (FedMVC) [8], which aims to explore a more comprehensive clustering structure from unlabeled multi-view data distributed across multiple clients while preserving privacy [21] . Despite the demonstrated potential of FedMVC in distributed environments, the amalgamation of these technologies has not garnered sufficient attention due to three crucial challenges, as depicted in Figure 1 .(1) Challenge 1: Due to the heterogeneity of local data, randomness in the training process, and differences in training time and computational resources among clients, there are inconsistencies in the feature representations encoded by the clients. Specifically, data representations on each client may exhibit angular deviation [51] , so an effective method is needed to align these representations.(2) Challenge 2: Heterogeneity intensifies this misalignment, as data from different clients may vary in terms of categories and quantities. While some methods [26, 28, 30, 53] address the challenge of non-independent and identically distributed (Non-IID) data in federated learning, they fall short when it comes to handling unsupervised learning tasks, making them difficult to apply directly.(3) Challenge 3: Multi-view data may be distributed across multiple clients, resulting in non-uniformity between the feature spaces encoded by each client. Therefore, efficiently performing multi-view information mining across multiple clients is challenging.Our solution: To address the aforementioned challenges, we propose HFMVC, a Heterogeneity-aware Federated deep Multi-View Clustering method. HFMVC is designed for distributed scenarios where each client possesses data from a single view. Specifically, each client learns representations specific to a single view by deploying an autoencoder to encode and reconstruct its local data. Then we map the encoded features from all views into a unified high-level space to eliminate irrelevant information. Based on this, we address the aforementioned challenges through the following methods: (1) To tackle the misalignment between representations generated by different clients mentioned in Challenge 1, we employ selective aggregation, wherein the server aggregates only the local models uploaded by clients with the same view data at a given time. This allows knowledge to be shared among these clients, improving their ability to represent data from the same view and aligning their encoded features in the feature space. (2) To mitigate the negative impact of heterogeneous data on feature representation mentioned in Challenge 2, our objective is to reveal the differences in feature representations across different heterogeneous environments and adopt targeted feature learning approaches. By evaluating the within-cluster sum of squares (𝑊 𝐶𝑆𝑆) and the average inter-cluster distance (𝐴𝐼𝐶𝐷) of the concatenated high-level features, we can measure the degree of heterogeneity and assess the training trends of local models, thus conducting model-level contrastive learning.  
federated unsupervised representation: 
 Unsupervised learning methods can be classified into generative and discriminative types. The former focuses on learning representations by generating pixels in the input space [18, 19, 27] , while the latter involves modeling data to discover intrinsic patterns or relationships within it [1, 16, 36] . Some related studies [24, 40, 40] have attempted to apply FL to unlabeled data. However, a straightforward combination of FL with unsupervised methods may not yield satisfactory results. Lu et al. [33] proposed FedUL, a federated unsupervised learning framework. But FedUL requires specific prior knowledge, which often poses challenges in practice. Liao et al. [31] proposed FedU 2 to analyze representation in federated unsupervised learning with Non-IID data. Zhang et al. [51] proposed FedCA, a method based on federated contrastive averaging with a dictionary and alignment. While FedCA overlooks privacy concerns and shows limited improvement in experimental results. Besides, there is limited research on the data representation problem in federated multi-view scenarios involving multiple clients. 
methodology 3.1 problem definition: 
 Here we present the formal definition of federated multi-view clustering. Suppose there is a dataset with 𝑁 samples and 𝑀 views distributed among 𝐶 clients (denoted as X = X 1 , X 2 , ..., X 𝐶 ), with an expectation to be partitioned into 𝐾 clusters. Each client possesses a subset of the data corresponding to a particular view X 𝑐 ∈ R 𝑁 𝑐 ×𝐷 𝑚 , where 𝑁 𝑐 and 𝐷 𝑚 represent the number of samples in client 𝑐 and the dimensionality of samples in view 𝑚, respectively. The local models with corresponding local data on the clientside are defined asF = {𝐹 1 (X 1 ; w 1 ), 𝐹 2 (X 2 ; w 2 ), • • • , 𝐹 𝐶 (X 𝐶 ; w 𝐶 )}.We use widely adopted autoencoders [9, 23, 44, 47, 48] for data representation and reconstruction. Specifically, each local model consists of the most basic encoder-decoder pair {𝑓 𝑐 𝜙 𝑐 (•), 𝑔 𝑐 𝜃 𝑐 (•)}, which denote the encoding and decoding processes for client 𝑐, where the learnable parameters are represented by 𝜙 𝑐 and 𝜃 𝑐 , respectively. Autoencoders can map raw data to a specified feature space, i.e., 𝑓 𝑐 𝜙 𝑐 (X 𝑐 ; 𝜙 𝑐 ) :X 𝑐 ∈ R 𝑁 𝑐 ×𝐷 𝑚 ↦ -→ Z 𝑐 ∈ R 𝑁 𝑐 ×𝑑 𝑚 and 𝑔 𝑐 𝜃 𝑐 (Z 𝑐 ; 𝜃 𝑐 ) : Z 𝑐 ∈ R 𝑁 𝑐 ×𝑑 𝑚 ↦ -→ X𝑐 ∈ R 𝑁 𝑐 ×𝐷 𝑚 .Here, 𝑁 𝑐 is the number of samples for client 𝑐, and 𝑑 𝑚 is the dimensionality of the features after encoding. Please note HFMVC assumes each client only has data from a single view, yet it still supports more scenarios by deploying multiple autoencoders on each client (please see the Appendix). Based on this, we introduce two additional definitions:(1) Friend clients: Refers to clients whose samples correspond to the same object but come from different views (e.g., client 1 and client n-1 in Figure 2 are friend clients).(2) Peer clients: Refers to clients whose samples correspond to different objects but come from the same view (e.g., client n-1 and client n in Figure 2 are peer clients).We instruct each client to reconstruct their local data to obtain client-specific feature representations. The reconstruction loss is:L 𝑅 = 𝐶 ∑︁ 𝑐=1 L 𝑐 𝑅 = 𝐶 ∑︁ 𝑐=1 X 𝑐 -X𝑐 2 2 = 𝐶 ∑︁ 𝑐=1 𝑁 𝑐 ∑︁ 𝑖=1 x 𝑐 𝑖 -𝑔 𝑐 𝜃 𝑐 𝑓 𝑐 𝜙 𝑐 x 𝑐 𝑖 2 2.(1) Due to the potential noise in the encoded features from individual client data and the possibility that data from different clients might be encoded into different latent spaces, we need to further learn high-level features within the same feature space. Therefore, we add an additional MLP layer P (𝑐 ) to the autoencoder model of client 𝑐, aiming to explore more valuable information. Here, we use H = H 1 , H 2 , ...,The objective of HFMVC is to achieve more comprehensive information acquisition through collaboration among multiple clients.Based on this, HFMVC performs selective aggregation among peer clients and dual contrastive learning among friend clients.Since the data of peer clients only vary in quantity and category, the server can selectively aggregate their local models to enhance each client's feature representation capabilities, thereby mitigating the negative impact of heterogeneous data. Thus the global model for view 𝑚 is:w𝑚 = 1 |C 𝑚 | ∑︁ 𝑖 ∈ C 𝑚 w 𝑖 .(2)Given that data among friend clients originates from different views but describes the same object, extracting complementary information between them proves advantageous for uncovering crucial semantics. Inspired by [45] , we implement contrastive learning among friend clients. Specifically, the server forwards the high-level features uploaded by client 𝑐 to its friend client(s). Following this procedure, each client acquires the high-level features of its friend client(s).Then each high-level feature h 𝑐 𝑖 from client 𝑐 generates (𝑀𝑁 𝑐 -1) feature pairs, denoted as {h 𝑐 𝑖 , h𝑝 𝑗 } 𝑝=1,...,|𝐶 ′ | 𝑗=1,...,𝑁 𝑐, where 𝐶 ′ represents the set of friends for this client, satisfying |𝐶 ′ | = 𝑀. And these pairs include (𝑀 -1) positive pairs {h 𝑐 𝑖 , h 𝑝 𝑖 } 𝑐≠𝑝 and the remaining 𝑀 (𝑁 𝑐 -1) negative pairs. Besides, cosine similarity is commonly employed as a measure of similarity for high-level features [45, 46] :EQUATIONWe use 𝜏 𝐶 to denote a temperature parameter, thus the overall contrastive loss for the clients possessing data from view 𝑐 and those possessing data from view 𝑝 is:... A little cat is comfortably yawning.A little dog is joyfully bouncing on the grass. = -1EQUATIONSince all clients perform the above steps, from a global perspective, the total contrastive loss can be accumulated as the sum of contrastive losses for all clients, and it can be represented as:EQUATIONBy conducting contrastive learning among friend clients, HFMVC achieves the exploration of common semantics across views. 𝜙 (X 𝑐 ; 𝜙) , H 𝑐,𝑡 = P (𝑐,𝑡 ) 𝑓 𝑐,𝑡 𝜙 𝑐 (X 𝑐 ; 𝜙 𝑐 ) and H 𝑐,𝑡 -1 = P (𝑐,𝑡 -1) 𝑓 𝑐,𝑡 -1 𝜙 𝑐 (X 𝑐 ; 𝜙 𝑐 ) . Then, we adopt different client-level contrastive learning strategies based on the system's degree of heterogeneity.(1) In IID scenarios, the sample distributions among peer clients are similar. However, due to factors such as randomness during training, different models may generate representations with certain angular deviations, resulting in misalignments between representations. Nevertheless, the model obtained through selective aggregation integrates knowledge from multiple clients, leading to a model with higher representational capacity. Therefore, we propose defining the model-contrastive loss:EQUATIONHere, 𝜏 𝑇 represents the temperature parameter. The significance of Eq. ( 6) is to treat (H 𝑐,𝑡 , H 𝑔,𝑡 ) as a positive pair and (H 𝑐,𝑡 , H 𝑐,𝑡 -1 ) as a negative pair. This allows the model to bring local representations closer to global representations, and amplify the changes in feature representations before and after each round of local training.( Figure 3 : Motivation of the heterogeneity-aware module. We observed that after pre-training, the concatenated features exhibit relatively poor (good) intra-cluster compactness and inter-cluster separability in the IID (Non-IID) scenarios.Therefore, the overall model-contrastive loss is the sum of individual contrastive losses across all clients:EQUATION)To precisely quantify the boundaries between IID and Non-IID for adaptation to Eq. ( 8), we design a heterogeneous-aware module. Indeed, the design motivation for this module stems from the following key observation (as shown in Figure 3 ): Key Observation: During the pre-training phase, as heterogeneity increases, the feature representations obtained by clustering local data on the client side become more similar, resulting in better separability between clusters and compactness within clusters for the global feature concatenated from all features' sets.Based on this, at the end of the pre-training phase, we apply k-means [34] to the concatenated high-level features. Then we compute the within-cluster sum of squares (𝑊 𝐶𝑆𝑆) and the average inter-cluster distance (𝐴𝐼𝐶𝐷) to assess the inter-cluster separability and intra-cluster compactness of the clustering structure: for 𝑐 = 1 to 𝐶 do in parallel 5:EQUATIONEQUATIONObtain high-level features and the global model.while not reach the maximum iterations 𝐸 do 7:Optimize the total loss function by Eq. ( 13). Exchange high-level features among friend clients.12:Distribute aggregated models selectively to each client. 13: end while 14: Calculate the clustering predictions by Eqs. ( 14)-( 15).Here, we use I(•) to represent the indicator function, and the complete L 𝑇 is expressed as:EQUATIONEq. ( 12) indicates the structure and value of the model-contrastive loss are correlated with the level of heterogeneity in the environment, where the quantification of heterogeneity is implemented by the heterogeneity-aware module.In summary, our objective function can be summarized as:EQUATIONwhere L 𝑅 , L 𝐶 , and L 𝑇 represent the local reconstruction loss of all the clients, the contrastive loss between friend clients, and the model-contrastive loss arising from selective aggregation among peer clients, respectively. Meanwhile, 𝛼 and 𝛽 denote the corresponding trade-off coefficients. Finally, the server performs k-means on all the high-level features to obtain the global centroids U:EQUATION)Therefore, the final prediction result for sample 𝑖 is:EQUATION)Algorithm 1 outlines the execution flow for both clients and the server in HFMVC. Once the pretraining is completed, the server assesses the degree of heterogeneity to selectively apply model contrastive learning strategies. Then each client optimizes its local loss. In turn, the server selectively aggregates data from peer clients and shares high-level features with friend clients. These steps form a complete iteration that is repeated until convergence is achieved. Datasets. We conduct experiments on four datasets: MNIST-USPS [37] (5000 samples with 2 views), BDGP [4] (2500 samples with 2 views), Multi-Fashion [42] (10000 samples with 3 views) and Caltech-5V [13] (1400 samples with 5 views). To better simulate heterogeneous scenarios in the real world, data partitioning adopts the Dirichlet distribution [20] . We configure four heterogeneous environments: Dirichlet (0.5), Dirichlet (1.0), Dirichlet (10) , and IID. Here, smaller values within Dirichlet (•) denote greater heterogeneity (imbalances in classes and labels), which is used to evaluate the adaptability of HFMVC and comparative methods.Comparison Methods. We select the following six state-of-theart (SOTA) methods for comparison: DEMVC [43] , SDMVC [44] , MFLVC [45] , GCFAgg [46] , FedDMVC [8] and FCUIF [38] . It is worth noting that among the aforementioned methods, only FedDMVC [8] and FCUIF [38] are applied in a federated environment, while the others are centralized methods. To ensure a fair comparison to the greatest extent, we make simple modifications to the alignment to support a distributed environment. Given the scarcity of research on FedMVC, these modifications are inevitable. Even for FedDMVC and FCUIF, their targeted FL scenarios are limited. In contrast, HFMVC supports distributions with an unlimited number of clients/views. We also set the client-to-view ratio to 5, which means that in a dataset with 2 views, we set up 10 clients accordingly.Evaluation Metrics. We evaluate the clustering performance using three metrics: accuracy (ACC), normalized mutual information (NMI), and adjusted rand index (ARI). 
comparison results: 
 We conduct experiments of HFMVC alongside other comparative methods on four publicly available datasets, with results presented in Table 1 . By comparing the results under four different heterogeneous settings, we can make the following observations:(1) Our method (HFMVC) consistently demonstrates superior performance and a significant lead across various experimental environments. For example, on the MNIST-USPS dataset, HFMVC outperforms in terms of ACC, NMI, and ARI, achieving maximum leads of up to 64.91%, 64.39% and 79.06%, respectively.(2) Compared to traditional centralized algorithms, HFMVC achieves significant advantages. Furthermore, relative to distributed algorithms like FedDMVC and FCUIF, HFMVC stands out due to its compatibility with multiple clients, stronger cross-client knowledge sharing, and information mining capabilities. Particularly in IID scenarios, the clustering performance of FedDMVC and FCUIF nearly collapses (with ACC on MNIST-USPS being respectively only 26.80% and 23.77%, while HFMVC achieves 98.27%), demonstrating the practical feasibility of only HFMVC.(3) HFMVC exhibits superior robustness. For instance, when the degree of heterogeneity ranges from Dirichlet(0.5) to IID, HFMVC's ACC on MNIST-USPS ranges from 86.21% to 98.27%, while on Caltech-5V, it ranges from 60.24% to 66.76%. In contrast, other methods exhibit much more significant fluctuations. It is particularly noteworthy that while HFMVC's overall performance tends to improve as the level of heterogeneity decreases, the situation is the opposite for the comparative methods. This seemingly paradoxical phenomenon can be attributed to the following: unlike centralized experimental environments, in distributed settings, data is partitioned into more client nodes, resulting in fewer data points being reconstructed by each autoencoder. As the level of heterogeneity increases, the data reconstructed by single autoencoders becomes purer (with fewer categories), thus yielding better results. HFMVC enhances its clustering performance in IID scenarios through selective aggregation and dual contrastive learning.We conduct comprehensive ablation studies on BDGP to evaluate the impact of each component, including L 𝑅 (defined in Eq. ( 1), L 𝐶 (defined in Eq. ( 5)), and L 𝑇 (defined in Eq. ( 12)), with results presented in Table 2 . And we can derive the following observations: (1) An intuitive result is that the method incorporating all components exhibits the best performance, while the result produced solely using L 𝑅 is the poorest, as it only focuses on data reconstruction. This holds true across all levels of heterogeneity.(2) The configuration lacking L 𝐶 performs worse compared to the setting where all components are used. This discrepancy becomes more pronounced, especially in settings with higher levels of heterogeneity. For example, the difference in ACC between these two settings is 13.04% under Dirichlet(0.5), while it reduces to 1.71% under Dirichlet(∞). This is because contrastive learning among clients aligns the representations of the same sample across different views. In other words, the absence of L 𝐶 contributes to poorer clustering results due to the lack of information exchange among friend clients (defined in Section 3.1).(3) The absence of L 𝑇 prevents the global model obtained through selective aggregation from guiding the training of local models on clients. This leads to two issues: firstly, when the degree of heterogeneity is low, the local training process fails to converge quickly, and peer clients (defined in Section 3.1) struggle to achieve maximum knowledge sharing. Secondly, when the degree of heterogeneity is high, due to imbalanced samples among clients, there exists significant diversity in the representational capabilities of each autoencoder. However, lacking L 𝑇 often results in a weakened representational capability of local data for each client.We conduct a detailed analysis of the hyperparameters in HFMVC, including 𝛼, 𝛽 (defined in Eq. ( 13)) and 𝜆 (defined in Eq. ( 11)). Specifically, this comprises two aspects: (1) Figures 4 (a)-(b ) illustrate the impact of 𝛼 and 𝛽 on clustering performance in IID and Non-IID scenarios. It can be observed that the clustering results exhibit overall high ACC values in IID scenarios, indicating good robustness (except when 𝛼=0.01 and 𝛽=0.001). In Non-IID scenarios, the best performance is observed when both 𝛼 and 𝛽 are within the range of 0.1 to 1. (2) Figures 4 (c )-(d) illustrate the impact of 𝜆 on clustering performance in IID and Non-IID scenarios. In fact, as a threshold coefficient, 𝜆 controls the boundary of heterogeneity in the system. Therefore, a smaller/larger value of 𝜆 may lead to inaccurate perceptions of heterogeneity, thereby affecting clustering performance. It can be observed that when 𝜆 is 5, all metrics achieve the best performance, regardless of whether it is in IID or Non-IID scenarios. It is worth noting that the value of 𝜆 varies depending on the dataset.Figure 5 illustrates the convergence curves of HFMVC under different levels of heterogeneity. It can be observed that under both conditions, HFMVC achieves rapid convergence and eventually stabilizes, indicating its excellent convergence properties. More importantly, the characteristic of rapid convergence significantly reduces the communication overhead of HFMVC.To measure the scalability of HFMVC, Figure 6 shows the clustering performance of HFMVC as the number of clients changes in environments with varying degrees of heterogeneity. We choose FedDMVC as the comparison method. It can be observed that: (1) HFMVC outperforms FedDMVC under all heterogeneity settings;(2) In IID scenarios, HFMVC exhibits excellent stability, as increasing the number of clients has little impact on its performance, whereas FedDMVC's performance significantly decreases; (3) In Non-IID scenarios, HFMVC's performance decreases as the number of clients increases, but it remains generally more stable than FedDMVC. These observations demonstrate that HFMVC possesses good scalability, mainly because HFMVC promotes the flow of knowledge among clients through selective aggregation and dual contrastive learning, preventing the clustering results from drastically declining with the increase in the number of clients. 
conclusion: 
 In this paper, we propose a Heterogeneity-aware Federated deep Multi-View Clustering (HFMVC) method, which enables the sharing of multi-view knowledge across clients while preserving privacy. Each client uses autoencoders to perform representation learning, followed by dual contrastive learning between the local and global models as well as among clients to explore consistent and complementary information. Besides, we design a heterogeneity-aware module that adaptively handles different heterogeneous scenarios. Extensive experiments validate that HFMVC exhibits the best performance in both IID and Non-IID environments. 
 
 Appendix:  


Decision: The paper presents a novel federated multi-view clustering framework, HFMVC, which addresses key challenges in multi-view clustering algorithms. The proposed method effectively handles inconsistent feature representation, integrates multi-view information, and adapts to heterogeneous data environments. The experimental results are extensive, demonstrating significant improvements over existing methods and illustrate the method's robustness and applicability.
Despite some areas that could benefit from further clarification and expansion, the reviewers appreciated the authors' rebuttal and argued for acceptance of the paper.

Reviews: review: paper_summary: paper_summary: The paper propose a novel federated deep multi-view clustering framework to protect local data privacy. The federated setting of this method has wide application scope, Moreover, a heterogeneity-aware module is proposed to the degree of data heterogeneity, while suit iid and non-iid data.  The paper has done extensive experiments to demonstrate its performance.
strength_weakness: strength_weakness: 1、	The motivations of the methods are clear and it is easy to understand through the Figure. It is solved to a certain extent through the novel clustering framework.
2、	The experiments section is sufficient and can verify the contribution.
3、	This federated setting that each client owns some sample of a single view has wide application.
limitations: limitations: 1	I think WCSS and AICD help to get larger distances between clusters and tighter distances within cluster, but how does it work as a Heterogeneity-Aware Module? Besides, the construction of Eq.12 is unclear.
2	Is the federated setting of your chosen compared federated multi-view methods same to your method?
3	Could you please further explain the difference between Eq.6 and Eq.7 and the reason why the positive pairs and negative pairs of iid and non-iid are different?
paper_summary: paper_summary: This paper generalizes multi-view clustering to federated learning settings, focusing on addressing heterogeneity and unevenness among data. Although not many new methods have been proposed in terms of multi-view clustering, it has solved some problems in the scenario of federated multi-view clustering. The authors divide clients into peer clients and friend clients. Selective aggregation is applied to peer clients while contrastive learning is conducted for friend clients. Experiments are ample and analysis is accurate, and the method proposed in this paper is far ahead of the comparative methods.
strength_weakness: strength_weakness: 1.This paper is detailed and requires a large amount of work
2.The authors acknowledge the challenges present in current federated multi-view clustering algorithms, such as inconsistent feature representation, difficulty in integrating multi-view information, and inability to adapt to heterogeneous data environments. 
3.HFMVC adopts strategies like contrastive learning and selective aggregation, enabling adaptive perception of heterogeneity and corresponding adjustments in clustering strategies.
limitations: limitations: 1.The innovation of the article is insufficient, and the methods used were proposed in the previous work. Having only one Heterogeneity Aware Module seems insufficient to reflect novelty.
2.The use of WCSS and AICD in the Heterogeneity Aware Module to quantify the boundary between IID and non IID is a bit abrupt. Why did not other clustering indicators be considered, or did mathematical proof or experimental analysis be provided.
3.The baseline of the article experiment itself is not applicable to the federated learning paradigm, and modifications to these methods have not been provided. This comparison may be unfair, which raises doubts about the experimental results. Can author consider integrating multi view methods and federated learning methods as a baseline.
4.In the experimental part, the adopted MNISTUSPS and BDGP have only 2 views, which seems a bit fewer. Moreover, the majority of experiments are based on these two data sets, the results may not be as persuasive as those from datasets with multiple views. Among the six compared methods, only two are in the federated learning environment, it is recommended to increase this number.
paper_summary: paper_summary: In this paper, the authors propose a heterogeneity-aware federated deep multi-view clustering method. It adaptively perceives the degree of heterogeneity and employs contrastive learning to explore consistency and complementary information across clients' multi-view data. Additionally, it seeks consensus among clients whose local data originates from the same view. Furthermore, the authors explain the sample representation logic for local clustering in various heterogeneous environments, propose two metrics to determine the degree of heterogeneity.
strength_weakness: strength_weakness: 1. The experimental results in the paper are extensive and show significant improvement over existing methods, demonstrating the superiority of the approach and making a substantial contribution to federated multi-view learning.

2. The method's illustrations are clear, allowing for an easy understanding of the method's structure.

3. The method introduces a noteworthy phenomenon and proposes two metrics based on this phenomenon, contributing to the analysis of IID and non-IID characteristics.
limitations: limitations: 1. There seem to be some issues with the legend. The definitions of peer clients and friend clients in Figure 2 appear to differ from those in the text, making it difficult for me to understand their meanings. Additionally, in Representation Learning, are all these features vectors with values 0 and 1? I think these vectors may not be suitable for autoencoders.

2. In my understanding, the optimization of clients is performed simultaneously according to Equation 13. However, in the equation, \(L_C\) seems to require the participation of friend clients, which suggests that friend clients are also being optimized. How is their parallelism ensured? Furthermore, conducting contrastive learning on clients requires substantial computational resources. Considering that clients are generally mobile devices, is it necessary to take into account the computational consumption on the clients?

3. The performance gap between the proposed method and other methods is too large. Could you further explain the specific reasons for this?

4. In the ablation study, there is a variant that removes (L_R). I believe removing this loss function might render the autoencoder ineffective. Are there any other modifications in this variant besides removing (L_R)?
paper_summary: paper_summary: The paper addresses the challenge of clustering analysis in multi-view data, particularly in the context of federated learning where data is distributed across multiple clients. Traditional Multi-View Clustering (MVC) methods assume centralized data storage, which is impractical and raises privacy concerns in real-world scenarios. Federated Learning (FL) offers a solution by aggregating models from different clients without sharing raw data, but integrating FL with MVC to form Federated Multi-View Clustering (FedMVC) introduces new challenges, including data heterogeneity, misalignment of feature representations, and efficient multi-view information mining.

To tackle these issues, the authors propose HFMVC, a Heterogeneity-aware Federated deep Multi-View Clustering method. HFMVC is designed for scenarios where each client holds data from a single view. It employs autoencoders to encode and reconstruct local data, mapping encoded features into a unified high-level space to eliminate irrelevant information. The method addresses the challenges through:
    1.Selective Aggregation :Aggregates local models from clients with the same view data to align feature representations and improve data representation.
    2.Heterogeneity Mitigation: Uses within-cluster sum of squares (WCSS) and average inter-cluster distance (AICD) to measure heterogeneity and guide model-level contrastive learning.
    3.Feature Learning Across Clients: Facilitates the exchange of high-level features via a trusted server and employs dual contrastive learning to enhance multi-view feature learning.
The proposed HFMVC demonstrates robust generalization capabilities, adaptability to both IID and Non-IID scenarios, and state-of-the-art performance in extensive experiments, all while ensuring data privacy. The contributions of this work include the novel federated MVC method, the introduction of a heterogeneity-aware module, and comprehensive experimental validation.
strength_weakness: strength_weakness: 1.Integrated Understanding of Information: By perceiving the same object from multiple perspectives, it helps to achieve a more comprehensive understanding of the information. The Multi-View Clustering (MVC) method can explore both consistent and complementary information in the data by promoting collaboration between multiple perspectives, thus achieving a comprehensive clustering structure.

2.Handling Data Heterogeneity: The HFMVC method can assess data heterogeneity and adaptively adjust clustering strategies based on local clustering results. This makes it highly adaptable to both Independent and Identically Distributed (IID) and Non-Independent and Identically Distributed (Non-IID) scenarios.

3.Efficient Feature Learning: By using a trusted server to facilitate the exchange of high-level features and employing a dual contrastive learning module, HFMVC can efficiently perform multi-view feature learning across multiple clients, ultimately uncovering a clearer clustering structure.

4.Experimental Validation: HFMVC has undergone extensive experiments with varying degrees of heterogeneity and different numbers of clients. The results show that HFMVC consistently exhibits state-of-the-art performance while maintaining privacy, demonstrating its effectiveness and robustness in distributed environments.
limitations: limitations: 1.Quantify the specific parameters of the transmitted data.
2.How multimodal datasets are specifically divided across different clients.