Federated Fuzzy C-means with Schatten-p Norm Minimization

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 15:13:12

Abstract: Federated multi-view clustering aims to provide a feasible and effective solution for handling unlabeled data owned by multiple clients. There are two main challenges: 1) The local data is always sensitive, thus preventing any inadvertent data leakage to the server or other clients. 2) Multi-view data contain both consistency and complementarity information, necessitating thorough exploration and utilization of these aspects to achieve enhanced clustering performance. Fully considering the above challenges, in this paper, we propose a novel federated multi-view method named Federated Fuzzy C-Means with Schatten-p Norm Minimization(FFCMSP) which is based on Fuzzy C-Means and Schatten p-norm. Specifically, we utilize the membership degrees to replace conventional hard clustering assignment in K-means, enabling improved uncertainty handling and less information loss. Moreover, we introduce a Schatten p-norm-based regularizer to fully explore the inter-view complementary information and global spatial structure. We also develop a federated optimization algorithm enabling clients to collaboratively learn the clustering results. Extensive experiments on several datasets demonstrate that our proposed method exhibits superior performance in federated multi-view clustering.

Main paper: 
 introduction: 
 Multi-view data refers to data containing information from multiple perspectives (e.g., modalities, sources, and viewpoints), and each view represents a distinct observation of the same object, thus, they generally have consistent and complementary information, which is beneficial for several applications. For example, by leveraging diverse types of medical data, such as CT scans and electrocardiograms, hospitals can obtain a more comprehensive assessment of a patient's physical condition. However, multi-view data learning confronts several challenges because the high cost of annotating complex multi-view data makes it difficult to acquire a substantial number of high-quality labels. Thus, multi-view clustering has attracted considerable attention as an unsupervised learning approach [5, 35] .Existing multi-view clustering methods can be summarised into two categories [12] : neural network-based multi-view clustering and heuristic-based multi-view clustering. Furthermore, neural networkbased multi-view clustering can be categorized into deep representation learning [40, 43] and deep graph learning [44, 45] while heuristic-based multi-view clustering can be categorized into nonnegative matrix factorization [19, 30] , graph learning [20, 28] , latent representation learning [22, 46] , and tensor learning [33] .Even though existing multi-view clustering methods achieve promising performance, they cannot be applied to scenarios where multi-view data is distributed across multiple clients, while clients are unwilling to share their data for the sake of privacy. Inspired by federated learning, which enables multiple clients to cooperatively train a global model without revealing any sensitive information [34] , federated multi-view learning is developed. For example, Huang et al. extended multi-view clustering based on non-negative matrix factorization (NMF) to construct a federated multi-view clustering method named FedMVL [19] by dividing the global optimization problem into multiple sub-problems that can be solved locally to fully protect local data privacy.However, FedMVL cannot obtain optimal performance because the underlying K-means enforces the clustering result to assign a concrete cluster label to each label and thus suffers from sensitivity to outliers and . Besides, it separates feature extraction and clustering into two steps and results in performance degradation. For another, FedMVL first learns each view's feature via NMF separately and then fuses multi-view features for clustering. Thus, it is unable to exploit the inter-view spatial structure of multi-view data.To overcome these weaknesses, we propose a novel Federated Fuzzy C-Means with Schatten-𝑝 Norm Minimization (FFCMSP). Compared with FedMVL, FFCMSP replaces K-means with fuzzy C-Means. Since fuzzy C-Means directly learn soft assignment via membership degree matrix, it alleviates the sensitivity to outliers due to the hard assignment of K-means and caused by two-step clustering. Besides, inspired tenser-based methods [33, 48] , we construct a tensor from the membership degree matrix of each view and validate Schatten 𝑝-norm based loss on it to learn the inter-view spatial structure. We further develop a distributed optimization algorithm, which enables all clients to optimize the model locally and collaboratively. Specifically, our contributions are as follows:• We introduce fuzzy C-Means to federated multi-view clustering to overcome the weaknesses of K-means. It utilizes the membership degree instead of hard assignment to capture the overall structure of distributed data with heterogenic features. • We construct a membership degree tensor from each view's membership degree matrix, and by minimizing the Schatten 𝑝-norm based regularizer, it can better explore the relationship between views and the complementary information of different views.  
related work 2.1 multi-view clustering: 
 Multi-view clustering aims to partition multi-view data into different clusters, which is widely applied in multi-view data labeling and pre-processing. Existing multi-view clustering methods include neural network-based multi-view clustering and heuristic multi-view clustering [12] . Although neural network based methods exhibit outstanding performance due to their ability to extract deep and nonlinear features of multi-view, their training cost is extremely high and is not suitable for lightweight applications. Differently, heuristic has lower computational complexity and better interpretability, which has attracted more attention recently. There are mainly four categories of heuristic multi-view clustering methods: non-negative matrix factorization-based (NMF) methods [19, 30] , graph learning-based methods [20, 28] , latent representation learning methods [22, 46] , and tensor learning-based methods [33] . The main idea of NMF-based methods is to factorize the data matrix of each view into a coefficient matrix and a basis matrix with lower dimensions for clustering. For example, Liu et al. [30] proposed joint multi-view NMF to learn a common consensus coefficient matrix; Liang et al. [29] introduced co-orthogonal constraints to capture the diversity within views and learn the orthogonal basis matrices. Multi-view graph clustering attempts to learn a consistent clustering structure from the correlation of samples, which is composed of graph fusion and graph partition. Wang et al. [41] firstly perceived the correlations between samples under the same view, and then all views adaptively collaborated to construct the consensus graph. Latent representation learning-based methods learn the shared latent representation of each view for clustering. [24] presents regularized and hybrid multiview coding (RHMC), which employs self-supervised learning to enhance the discriminative information of the shared feature.The aforementioned methods all process multi-view data separately and then fuse the learned features or clustering results. To better exploit the inter-view spatial structure, tensor-based methods compose multi-view data into a three-order tensor. For instance, Lu et al. [33] constructed the third-order tensor from view-specific label matrices and minimized the divergence between the matrices to fully explore the complementary information between views. Li et al. [23] extended NMF into orthogonal non-negative tensor factorization and introduced tensor Schatten 𝑝-norm regularizer to fully utilize the complementary information provided by each view. Nevertheless, these methods are designed for centralized settings and ignore the possibility that multi-view data are held by different entities.Federated learning [34] is a distributed learning paradigm that enables multiple entities to collaboratively train a global model on data distributed across multiple devices without revealing any private information. The concept of federated learning and its specific algorithm, FedAVG, were proposed simultaneously [34] . FedAVG involves performing stochastic gradient descent on each client while aggregating the local trained models on the server by performing averaging. Afterward, many variants of federated learning were proposed: FedAMP [21] , FedProx [27] . Federated learning can be categorized into three types: Horizontal Federated learning (HFL), vertical federated learning(VFL), and federated transfer learning (FTL).In HFL, several clients share the same feature spaces but different sample spaces. [14] introduced HFL for Electroencephalography classification techniques to meet the requirements of privacy protection and data heterogeneity. Moreover, [47] focused on measuring the contribution to the model of all clients and the global server evaluates the contribution through reinforcement learning techniques. For VFL, several clients share the same sample space but different feature spaces. [6] proposed a novel method that enables each client to execute stochastic gradient algorithms independently, and a new technique of perturbed local embedding is proposed. Traditional VFL identifies the shared sample by Private Set Intersection(PSI), which may lead to some data leakage. Thus, [39] proposed a Private Set Union(PSU) based VFL framework to avoid such kind of data leakage. For FTL, clients share both different sample spaces and feature spaces. [31] first proposed the concept of FTL, enabling a target-domain party to build flexible and effective models by leveraging rich labels from a source domain. All the previously mentioned methods are designed for single-view data and fail to fully exploit the latent information inherent in multi-view data for learning.Traditional multi-view clustering methods generally assume that all data are held by a single party and never consider the application in which multi-view data are distributed across different clients and privacy-preservation. To tackle this problem, federated multiview learning was proposed, and there are several federated multiview learning methods developed for different settings. Multi-view learning was introduced to the personalized recommendation system [10] , cancer subtype identification [7] , and other domains.However, similarly, labeling multi-view data is costly, several times more expensive than labeling single-view data. This led to the emergence of federated multi-view clustering(FMVC). [4] introduced FMVC to the medical field. The proposed method applies to both VFL and HFL and keeps sensitive data private. [8] is a federated multi-view clustering method based on neural networks and guarantees data security by transmitting non-sensitive parameters between clients and the global server. Neural network-based FMVC methods are always time-consuming. Thus, most FMVC methods are based on heuristic clustering methods. [19] combines NMF and K-means to construct a time-efficient FMVC framework and the final clustering result comes from the coefficient matrix. However, because of using K-means on the coefficient matrix, we cannot effectively capture the relationship between samples and clusters, leading X 𝑚 , V 𝑚 denotes the data matrix and center cluster centroid matrix. The grey ♦ represents that the membership matrix U is used to describe the membership relationship between samples and cluster centroid. Moreover, black ⟲ visually represents whether the parameters are updated on the client or the server. The contruction of tenser 𝒰 is also visualized. Finally, the Δ (𝑐 ) denotes the 𝑐-th frontier slice of 𝒰 and it well characterize the complementary information embedded in inter-views. [15] to some information loss and performance degradation. Therefore, [17] proposed fuzzy C-Means-based FMVC which uses membership degree instead of hard clustering assignment. Nevertheless, it fails to capture the consistent information between views. 
method 3.1 preliminaries: 
 For easier presentation, we introduce Schatten 𝑝-norm [32] as follows. DEFINITION 1. The matrix Schatten 𝑝-norm for matrix M ∈ R 𝑛 1 ×𝑛 2 is defined as below:||M|| 𝑠𝑝 = min(𝑛 1 ,𝑛 2 ) ∑︁ 𝑖=1 𝜎 𝑝 𝑖 (M) 1 𝑝 (1)where 𝜎 𝑖 (M) is the 𝑖-th singular value of matrix M.Given a tensor ℳ ∈ R 𝑛 1 ×𝑛 2 ×𝑛 3 , then we have its t-SVD:ℳ = 𝒰 * 𝒮 * 𝒱 𝑇 (2)where 𝒰 ∈ R 𝑛 1 ×𝑛 1 ×𝑛 3 , 𝒱 ∈ R 𝑛 2 ×𝑛 2 ×𝑛 3 are both orthogonal tensors, and 𝒮 ∈ R 𝑛 1 ×𝑛 2 ×𝑛 3 is diagonal tensor. Moreover, all frontal slices of S are diagonal matrixes, and the 𝑖-th singular value of 𝑘-th frontal slices of 𝒮 is denoted as 𝒮 𝑘 (𝑖).Based on Definition 1 and Definition 2, the tensor Schatten 𝑝-norm for tensor ℳ is defined as below:||ℳ|| 𝑠 𝑝 = min(𝑛 1 ,𝑛 2 ) ∑︁ 𝑖=1 𝑛 3 ∑︁ 𝑘=1 𝒮 𝑘 (𝑖) 𝑝 1 𝑝 (3)where 0 ≤ 𝑝 ≤ 1, tensor Schatten 𝑝-norm has proven to be a better rank approximation method than the tensor nuclear norm (TNN), which motivates us to apply it in our work to learn more robust and clustering results. Given a typical Tensor Schatten 𝑝-norm problem, it can be solved with the following Lamma.EQUATION)has the optimal solution given byℬ * = ifft 𝒰 * 𝒟 𝜏,𝑝 (𝒞) * 𝒱 𝑇 (5)where U and V are obtained via t-SVD of 𝒞, i.e. 𝒞 = 𝒰 * 𝒮 * 𝒱 𝑇 , and𝒟 𝜏,𝑝 C (𝑚) = 𝑑𝑖𝑎𝑔 𝜁 C (m) , 𝜁 C (m) = GST 𝜎 S (m) , 𝜏, 𝑝 . The GST algorithm is introduced in [15].Problem Statement: We consider the same federated multi-view settings as that in [19] , which includes a centralized server S and 𝑀 clients. Multi-view data, denoted as X = {X (1) , ) is the feature dimension, 𝑁 is sample number, and 𝑀 is the number of views as well as client number. Our goal is enable the server and clients to cooperatively train a global clustering model without exchanging any raw data. Federated Multi-View Fuzzy C-Means: Our work is derived from fuzzy C-Means (FCM) [1] , which assigns data points to clusters with membership degree matrix. Suppose there are 𝑁 samples of dimension 𝐷 denoted by X ∈ R 𝑁 ×𝐷 , FCM divides them into 𝐶 clusters with the following objective:X (2) , • • • X (𝑀 ) }, is held by different clients. Specifically, the 𝑚-th view X (𝑚) ∈ R 𝑁 ×𝑑 (𝑚) (𝑚 = 1, 2, • • •, 𝑀) is distributed in the 𝑚-th client, C 𝑚 , where 𝑑 (𝑚EQUATIONwhere 𝑢 𝑖 𝑗 is the membership degree, representing the probability that the i-th sample belongs to the j-th cluster; V ∈ R 𝐶 ×𝐷 is the clustering centroids matrix; 𝑡 is the fuzzification coefficient, and bigger 𝑡 tends to defocus membership towards the fuzziest state [1] , whose suggested range is 1 ≤ 𝑡 ≤ 3.Compared with K-means, FCM can better resist noises or outliers, motivated by which, we adopt multi-view FCM as our underlying multi-view clustering method, which can be expressed as follows: where X (𝑚) is the data matrix located on the 𝑚-th clients and V (m) is the corresponding cluster centroids. 𝑡 is also the fuzzification coefficient, similarly. To simplify the optimization process, this paper sets the fuzzification coefficient 𝑡 to 1. Thus, based on Eq. (7), the objective function is:min 𝑈 (𝑚) ,𝑉 (𝑚) 𝑀 ∑︁ 𝑚=1 𝑁 ∑︁ 𝑖=1 𝐶 ∑︁ 𝑗=1 𝑢 (𝑚) 𝑖 𝑗 𝑡 ||x (𝑚) 𝑖 -v (𝑚) 𝑗 || 2 𝐹 𝑠.𝑡 . 𝐶 ∑︁ 𝑗=1 𝑢 (𝑚) 𝑖 𝑗 = 1, 𝑢 (𝑚) 𝑖 𝑗 ≥ 0, 𝑡 ≥ 1 (7)EQUATIONTensor Schatten 𝑝-Norm Based Regularizer: By observing Eq. (8), we find that in each view, each sample corresponds to a membership degree 𝑢 𝑖 𝑗 for each cluster, and for 𝑁 samples and 𝐶 clusters, the membership degrees form a matrix U (𝑚) ∈ R 𝑁 ×𝐶 , thus latent information is hidden between different view-specific membership degree matrix. Inspired by [33] , we introduce a regularizer based on the tensor Schatten 𝑝-norm to enforce the model to learn the global spatial structure and inter-view complementary information, the final objective function becomes:minEQUATIONwhere 𝜆 is the trade-off parameter and the membership degree tensor 𝒰 is constructed by stacking all views' local membership degree matrix and taking a simple rotation as shown in Fig. 1 . 
optimization algorithm: 
 In centralized scenarios, Eq. (9) can be easily solved. However, with the multi-view data distributed to local clients, our design goal is to employ federated learning to collaboratively learn a clustering model while considering the requirement of data privacy. This requires that the raw models or parameters transmitted between clients and the global server should be non-sensitive, which poses extra challenges of optimization. To address this issue, we further develop a federated optimization algorithm in this scenario. Firstly, we introduced augmented Lagrange multiplier(ALM) to Eq. (9):EQUATION)where 𝜇 is the penalty parameter, 𝒬 is the Lagrange multiplier, and ⟨•⟩ denotes the inner product operation. The optimization algorithm is divided into three parts, as described below.Solving U (𝑚) with fixed 𝒢 and 𝒱: with fixed 𝒢 and 𝒱, the objective function becomes:EQUATIONConsidering the independence of 𝒰 , 𝒬, and 𝒢, Eq. (10) could be decomposed into 𝑀 independent subproblems, where the 𝑚-th subproblem is:EQUATIONThrough simple algebra, Eq. ( 12) can be transformed into:EQUATIONwhereEQUATION𝐹 , and T (𝑚) = G (𝑚) -Q (𝑚)𝜇 . By introducing the Karush-Kuhn-Tucker (KKT) condition, we have:EQUATIONSolving 𝒢 with fixed 𝒰 and 𝒱: With fixed 𝒰 and 𝒱, the objective function concerning 𝒢 becomes:EQUATION)where 𝒫 = 𝒰 -𝒬 𝜇 . From Lamma. 1, we get the optimal solution of Eq. (15):EQUATIONwhere 𝒰 and 𝒱 * are obtained via t-SVD of 𝒫, i.e., 𝒫 = 𝒰 * 𝒮 * 𝒱 𝑇 * . Solving V (𝑚) with fixed 𝒰 and 𝒢: With fixed 𝒰 and 𝒱, the objective function concerning 𝒱 becomes:min 𝑀 ∑︁ 𝑚=1 𝑁 ∑︁ 𝑖=1 𝐶 ∑︁ 𝑗=1 𝑢 (𝑚) 𝑖 𝑗 ||x (𝑚) 𝑖 -v (𝑚) 𝑗 || 2 𝐹 (17)Similarly, we decompose Eq. (17) into 𝑀 independent subproblems owing to the independence of 𝒰 , and the 𝑚-th subproblem is defined as: min L V (𝑚) = min 𝑁 ∑︁ 𝑖=1 𝐶 ∑︁ 𝑗=1 𝑢 (𝑚) 𝑖 𝑗 ||x (𝑚) 𝑖 -v (𝑚) 𝑗 || 2 𝐹 (18)Considering local client data security, the data or parameters transmitted between the client and the server must be insensitive. Therefore, based on the above optimization algorithm, we propose the workflow in the federated setting. By observing Eq. ( 14), Eq. ( 16), and Eq. ( 19), we draw the following conclusions:• The update of 𝑢 (𝑚) 𝑖 𝑗 is related to 𝑑 (𝑚)𝑖 𝑗 , G (𝑚) , and Q (𝑚) , where 𝑑(𝑚) 𝑖 𝑗describes the is between 𝑖-th sample and 𝑗-th cluster centroid and it can be calculated locally. Therefore, each client can update 𝑢 (𝑚) 𝑖 𝑗 locally as long as they have a backup of G (𝑚) and Q (𝑚) .• The update of 𝒢 is related to 𝒰 and 𝒬, thus, it can only be performed globally and U (𝑚) should be transmitted to the global server in advance. • The update of V (𝑚) is related to U (𝑚) and X (𝑚) , and it can be performed locally because the relevant data is stored locally. • The optimal solution of 𝒬 is 𝒬 ← -𝒬 + 𝜇 (𝒰 -𝒢), which can only be performed globally because it requires tensor U.After the above analysis, we described the detailed workflow in the federated setting.(1) Firstly, each client C 𝑚 initializes 𝑢(𝑚) 𝑖 𝑗 = 1𝐶 and V (𝑚) randomly, and the global server initializes 𝒢 = 𝒰 and 𝒬 = 0.(2) In each communication round, each client C 𝑚 updates U (𝑚) with Eq. ( 14) and V (𝑚) with Eq. ( 19) locally and then sends updated U (𝑚) to the global server S. (3) After receiving them, S stacks all U (𝑚) into a tensor 𝒰 , and then update 𝒢 with Eq. ( 16) and 𝒬 with 𝒬 ← -𝒬+𝜇 (𝒰 -𝒢).The global server then transmits the 𝑚-th slice of updated G and G, G (𝑚) and Q (𝑚) , to the corresponding client. (4) The step ( 2) and (3) repeat until convergence, and then, the final cluster assignment matrix H is calculated globally by:ℎ 𝑖 𝑗 =        1 𝑗 = arg min 𝑝 𝑀 𝑚=1 u (𝑚) 𝑖 𝑝 0 otherwise (20)The detailed workflow is summarized in Algorithm 1. Local client-side: The computation load on each client includes the update of U (𝑚) and V (𝑚) . In each local iteration, the computational complexity of the update of U (𝑚) is O 𝑁𝐶 𝑑 (𝑚) + 𝐶 while the computational complexity of the update of V (𝑚) is O (𝑁𝐶). Thus, Algorithm 1 Federated Fuzzy C-Means with Schatten 𝑝-Norm Minimization(FFCMSP)input: The data X = {X (1) , X (2) , ..., X (𝑀 ) } on 𝑀 local clients, cluster number 𝐶; parameters: 𝜇, 𝑝, 𝜆; output: Output: Cluster assignment H;1: Client initialization: Initialize 𝑢 (𝑚) 𝑖 𝑗 = 1𝐶 and V (𝑚) randomly. 2: Server initialization: Initialize 𝒬 = 𝒰 and 𝒢 to be all zero. 3: while not converged do Update membership matrix U (𝑚) according to (14); 7:Update cluster centroids matrix V (𝑚) according to (19) Stack all U (𝑚) to construct tensor 𝒰 as shown in Fig. 1;  12:Update 𝒢 with new 𝒰 according to (16); Send G (𝑚) and Q (𝑚) to C 𝑚 ; 15: end while 16: S aggregates U (𝑚) into cluster assignment matrix H according to (20); 17: return H the overall computational complexity of updating local model 𝐼𝑇 𝐸𝑅 𝐿 iterations is O 𝐼𝑇 𝐸𝑅 𝐿 𝑁𝐶 𝑑 (𝑚) + 𝐶 . Global client-side: The computational load on the global server includes the update of G and Q. In each global iteration, the computational complexity of the update of G is O 2𝑁𝐶𝑀log (𝐶𝑀) + 𝑁𝐶𝑀 2 , while the computational complexity of the update of Q is O (𝑁𝐶𝑀). Thus, the overall computational complexity of updated global model 𝐼𝑇 𝐸𝑅 𝐺 iterations is O 𝐼𝑇 𝐸𝑅 𝐺 2𝑁𝐶𝑀log (𝐶𝑀) + 𝑁𝐶𝑀 2 .Local Client-→Global Server: Each client transmits its local U (𝑚) to the global server before aggregating. For all clients, the amount of data transmitted in each communication round is O (𝑁𝐶𝑀). Global Server -→Local Client: The global server sends the 𝑚-th slice of G and Q to the m-th client after aggregating. Thus, the total amount of data transmitted in each communication round is O (2𝑁𝐶𝑀). 
experiment 5.1 experimental settings: 
 We validate our model on eight multi-view datasets and compare it with several state-of-the-art methods. We implement all the methods with MATLAB2023b on a Desktop running Windows 11 equipped with Intel Core i5-13400CPU(2.50GHz) and 32GB DDR4 RAM. For the federated setting, our experiment consists of the same number of clients as the number of views and one server, while each client stores the data with one view. We use three widely used metrics, i.e., Accuracy (ACC), Normalized Mutual Information(NMI), and Purity(PUR) to evaluate our method. [38] is about images of indoor scenes and corresponding descriptions. We preprocess the dataset by following [40] and obtained a two-view dataset of 1449 samples divided into 13 classes, and the feature dimensions are 2048, and 300.Compared Methods: We compared our methods with five centralized multi-view clustering methods and two federated multi-view clustering to verify the superiority of our proposed model. Concretely, (1)DiMSC [3] boosts the multi-view clustering by mining the complementary information among multi-view features.(2)MvLRSSC [2] constructs an affinity matrix shared among all views to learn a joint subspace representation. (3)RMSL [26] is composed of Hierarchical Self-Representative Layers and Backward Encoding Networks, which recover the subspace structure of data and explore the complex relationships among different views respectively. (4)GMC [42] is a graph-based multi-view clustering method that learns a unified graph from the affinity graph of each view for clustering. ( 5)MvDGNMF [25] is an NMF-based method that can extract more abstract representation by constructing a multi-layer NMF model with graph Laplacian regularizer. ( 6) UDBGL [11] jointly learns both single-view and consensus graphs and combines it with k-means to construct a unified framework. ( 7) FastMICE [18] presents the concept of a random view group to capture the versatile view-wise relationships and design a hybrid early-late fusion strategy. The method has linear time and space complexity. (8)FedMVL [19] is a federated multi-view clustering based on NMF and K-means. This could be attributed to the proposed FFCMSP effectively capturing the consistency and complementary information between views using tensor Schatten 𝑝-norm while leveraging FCM to better preserve information between data points. Besides, in the eight chosen compared method, FedMVL is the only federated multi-view clustering method, thus, the experiment results further demonstrate the the superiority of FFCMSP over centralized ones. Overall, these experimental results validate the superiority of FFCMSP in multi-view clustering tasks.We record the values of The objective function Eq. ( 9) has two main parameters: 𝑝 and 𝜆.We analyze their impact on the clustering results, which are depicted in Fig. 3 . Firstly, we analyze the influence of two parameters. 𝑝 is the most significant parameter in the tensor Scatten-p norm. When 0 ≤ 𝑝 ≤ 1, the tensor Scatten p-norm has better rank approximation. 𝜆 is the trade-off parameter to balance the effects of the two items.When 𝜆 is large, the tensor Schatten 𝑝-norm dominates, leading the model to explore more spatial structural information between views, while when 𝜆 is small, the model focuses on local training. From Fig. 3 , we conclude that the chosen for 𝑝 is closely related to the dataset, but when 𝜆 is set to a larger value, the model performs better. Above all, the suggested range of 𝜆 is [100, 1000].Our proposed FFCMSP consists of two main modules: The fuzzy C-Means module and the tensor Schatten 𝑝-norm. As the tensor U is constructed from the all view-specific membership degree matrix U, thus the tensor Schatten 𝑝-norm cannot exist independently. Consequently, we conduct the ablation study in the following two cases: (1) FFCMSP w/o tensor Schatten-𝑝 regularizer and fuzzy C-Means(case 1); (2)FFCMSP w/o tensor Schatten 𝑝-norm regularizer(case 2); It is obvious that case 1 is equal to multi-view K-means, and the results are shown in Table . 3. From the results, we observe that on 3-sources, the ACC, NMI, and PUR of case 2 outperform that in case 1 by 7.28%, 18.57%, and 7.28 %, demonstrating that introducing membership degree to clustering surely enhances clustering performance. Moreover, the ACC, NMI, and PUR of FFCMSP on 3-sources also increase by 10.65%, 12.71%, and 10.65% compared with case 2, indicating that introducing the tensor Schatten 𝑝-norm can better explore spatial structural information between views. In summary, the ablation experiments strongly support the significant role of the two key modules in enhancing clustering performance. 
conclusion: 
 In this paper, we proposed a novel fuzzy C-Means and Schatten 𝑝norm based federated multi-view clustering method named FFCMSP. This method integrates fuzzy C-Means clustering and Schatten 𝑝norm regularizer to enhance the clustering performance of multiview data in a federated learning setting. Specifically, by employing fuzzy C-Means, we effectively alleviate the information loss caused by K-means, while the Schatten 𝑝-norm helps to exploit the inter-view complementary information and spatial structure, thereby improving the accuracy and robustness of clustering. Besides, the proposed federated optimization algorithm enables clients to train the global clustering model collaboratively. Furthermore, we evaluated the FFCMSP method on multiple real-world datasets and compared it with several state-of-the-art multi-view clustering methods. The experimental results demonstrate that our method generally achieves superior clustering performance.  
 
 Appendix:  


Decision: This paper proposes a method termed Federated Fuzzy C-Means with Schatten-p Norm Minimization (FFCMSP). The proposed method can tackle two key challenges in federated multi-view clustering: data privacy and the need to leverage both consistency and complementarity in multi-view data. This article has received 4 reviews. The 4 reviewers are confident. 2 reviewers reject this paper, and 2 reviewers accept it. I suggest to accept as the positive balance seems better.

Reviews: review: paper_summary: paper_summary: The research addresses the challenges of federated multi-view clustering. A new method called FFCMSP is proposed. This method improves traditional K-means by using fuzzy C-means to better manage uncertainty and reduce information loss. The incorporation of a Schatten 𝑝-norm regularizer helps to leverage inter-view complementary information and the global spatial structure of the data. A federated optimization algorithm is developed to allow clients to collaboratively train the model without data leakage. Experimental results on various datasets demonstrate that FFCMSP outperforms existing multi-view clustering methods in accuracy and robustness.
strength_weakness: strength_weakness: The paper is written in a clear and comprehensible manner, with a logical flow of ideas, and includes necessary theoretical proofs and experimental analyses.
limitations: limitations: 1. Reference [17] in the paper also uses the fuzzy C-means method for multi-view clustering. It appears that this paper merely adds a Tensor Schatten $p$-Norm but does not provide a comparative analysis with that method. Additionally, it mentions a comparison with two federated multi-view methods on line 681, but on line 798, it states that only one federated multi-view method was compared.

2. Is it that common federated learning methods cannot be directly applied to multi-view data? Why? And I am interested in how this proposed method compares with other non-federated learning algorithms.

3. The title of the paper does not indicate that this is a federated learning algorithm designed for multi-view data, thus failing to reflect the characteristics of the method.
paper_summary: paper_summary: The paper focuses on two problems in federated multi-view clustering. First, multi-view data might be distributed across different clients, which hinders the training of multi-view clustering model. For another, multi-view data contain both consistent and complementary information. To overcome these two problems, the authors proposed a novel federated multi-view method named Federated Fuzzy C-Means with Schatten-p Norm Minimization. This method adopts fuzzy C-means to alleviate the weakness of K-means. With the tensor schatten p-norm minimization, it can better exploit the inter-view information. A federated optimization algorithm is designed to support the training of the proposed multi-view fuzzy C-means without leaking any information.
strength_weakness: strength_weakness: 1.The proposed method can solve the weaknesses of multi-view K-means and can be trained by multiple clients without leaking the private local data.
2.The collaborative optimization algorithm is computationally efficient and does not require much memory.
3.The introduction of the tensorial constraint helps to explore the complementary information of different views.
4.The experimental results illustrate the effectiveness of the proposed method.
limitations: limitations: 1.For the tensor schatten-p norm used in the method, the parameter $p$ seems to have an important influence on the final performance. Especially, wheen $p=1$, the tensor Schatten p-norm is tensor nuclear norm, and when $p=0$, it is the tensor rank. However, the influence of $p$ in tensor Schatten p-norm on the model performance is not revealed in the experiment.
2.Figure 1 should be further improved and clarified. The diamond symbol introduced in the title of Figure 1 is not consistent with that in the figure. Besides, some information in the Algorithm 1 is not revealed in this Figure. For example, whether $Q^m$ is transmitted between clients and global server is not well illustrated.
3.For the experiments, the authors only compare the proposed method with one federated method, which may not be enough to evaluate the performance. Maybe more federated methods should be compared. Besides, since the authors argued that the proposed federated multi-view C-means could ensure data privacy, the security issue should also be analyzed or discussed.
paper_summary: paper_summary: In this paper, authors propose a federated fuzzy C-means for multi-view clustering. Its main contributions lie in that it first introduces a tensorized Schatten p-norm regularization to help the fuzzy C-means to better capture the inter-view spatial features embedded in multi-view data. In this way, the complementary information can be better learned by the clustering model, thus achieving better clustering performance. Furthermore, the authors proposed a federated fuzzy C-means, which well addresses the issue that the multi-view data is held by different parties and supports collaborative training without exchanging the raw data. The proposed method helps to extend the potential applications of multi-view clustering. To illustrate the superiority of the work, the paper compares it with both centralized and federated methods via several experiments.
strength_weakness: strength_weakness: 1. The paper is with a clear organization and is relatively easy to follow. 
2. The work explores the application of multi-view clustering under the federated scenarios and discusses a practical deployment problem of multi-view clustering. The provided multi-view fuzzy C-means method contributes to solving this problem with fuzzy C-means enhanced by tensorized schatten-p norm as well as carefully designed federated optimization algorithm. 
3. The paper is technically sound. Especially, the paper includes a formal derivation indicating how the optimization algorithm is developed.
4. Extensive experiments are conducted.
limitations: limitations: 1. There are some typos in this paper. For example, Figure 2 depicts the relationship between ACC/Objective and the number epochs. However, the description on Figure 2 mentions its is with regard to the number of iterations. The authors should current this inconsistency issue; Eq. (8) lacks the parameter to be optimized.
2. The section on related work could be improved by including more references to multi-view learning and federated multi-view clustering, particularly those on Schatten p-norm regularization. This would provide a more comprehensive background and show the broader applicability of the proposed method.
3. The federated workflow should be explained in more detail. It is suggested to integrate this description with Figure 1 to provide a clearer understanding of the workflow and how it operates in a federated learning scenario.
paper_summary: paper_summary: This paper proposes a method termed Federated Fuzzy C-Means with Schatten-p Norm Minimization (FFCMSP). The proposed method can tackle two key challenges in federated multi-view clustering: data privacy and the need to leverage both consistency and complementarity in multi-view data. Several experiments are conducted to verify the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The proposed method improves clustering accuracy.
2. The proposed method can protect data privacy.
limitations: limitations: 1. The paper has limited innovation. The proposed method appears to be merely a traditional multi-view clustering. I acknowledge that this method can handle federated learning tasks; however, its main innovation does not lie in this aspect.
2. Can you explain why the objective function is not monotonically decreasing? I suspect there are some errors in the implementation of the algorithm.
3. Some grammar errors or typos need to be corrected. "... sensitivity to outliers and "  ---> “sensitivity to outliers” on Line 89.
4. The writing of the paper needs improvement. The related work section should describe specific methods closely related to the proposed approach, rather than conducting another literature review.