DFMVC: Deep Fair Multi-view Clustering

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-06 09:35:50

Abstract: Existing fair multi-view clustering methods impose a constraint that requires the distribution of sensitive attributes to be uniform within each cluster. However, this constraint can lead to misallocation of samples with sensitive attributes. To solve this problem, we propose a novel Deep Fair Multi-View Clustering (DFMVC) method that learns a consistent and discriminative representation instructed by a fairness constraint constructed from the distribution of clusters. Specifically, we incorporate contrastive constraints on semantic features from different views to obtain consistent and discriminative representations for each view. Additionally, we align the distribution of sensitive attributes with the target cluster distribution to achieve optimal fairness in clustering results. Experimental results on four datasets with sensitive attributes demonstrate that our method improves both the fairness and performance of clustering compared to state-of-the-art multi-view clustering methods.

Main paper: 
 introduction: 
 With the rapid development of digital technologies, data often consists of diverse features or originates from various views. For example, an image can be described by diverse feature sets, including Local Binary Patterns (LBP), Histogram of Oriented Gradients (HoG) and Scale-Invariant Feature Transform (SIFT), and each feature set can represent a view of an image. Multi-view clustering (MVC), as an unsupervised learning task, has gained considerable attention in various machine learning applications, including data mining [50] , scene recognition [34] and information retrieval [15] . Compared to single-view clustering, because of the complementary nature of information between different views [5, 34] , MVC significantly enhances clustering performance by integrating information from multiple views. The goal of MVC is to divide data samples into several disjoint groups by using feature information between different views. Existing MVC algorithms can be roughly categorized into traditional MVC methods and deep MVC methods. Traditional MVC methods contain three categories: non-negative matrix factorization (NMF)-based methods [30] , graph-based methods [24, 26] , and subspace-based methods [53, 56] . NMF-based MVC employs matrix factorization to learn a shared representation for multi-view data. Graph-based MVC methods construct a unified graph structure using multi-view data. Subspace-based MVC methods aim to learn a consistent subspace representation. However, these traditional MVC methods primarily rely on linear transformations, limiting their potential to improve clustering performance.Deep MVC algorithms have been proposed to address the aforementioned limitation, leveraging the powerful nonlinear feature extraction capabilities of deep neural networks [11, 36, 48, 49] . These methods employ view-specific encoder networks to transform each view and derive a consensus representation. For example, MvDSCN [57] learned a multi-view self-representation matrix in an end-toend manner. Inspired by contrastive learning techniques, many multi-view clustering methods integrate this technique into the model. Most existing contrastive learning-based methods attempt to maximize the mutual information contained among the assignment distributions of multiple views. For instance, COMPLETER [29] learned an informative and consistent representation of multiview data by contrastive learning; DCP [28] conducted consistency learning and data recovery through dual contrastive prediction; MFLVC [49] introduces multi-level feature learning with a contrastive strategy for multi-view clustering, e.g., low-level features, high-level features and semantic features. Although these methods achieve remarkable clustering performance, it should be noted that multi-view data often contains sensitive attributes, which can influence the clustering outcome. For instance, if the sensitive attribute is gender, one cluster may predominantly consist of males while another cluster predominantly consists of females, resulting in gender bias. In Figure 1 (a), we can see that it will lead to unfair clustering.To alleviate the unfairness issue, numerous fair clustering methods have been proposed [10, 20, 22] . For instance, fair clustering was first proposed and divided the original data into several chunks with fairness constraints in [10] . In Figure 1 (a) and (b), fair clustering can ensure the uniform distribution of samples containing sensitive attributes in each cluster. SpFC [20] imposed linear fairness constraints on spectral clustering. DFC [22] exploited adversarial training to make the clustering results independent of sensitive attributes. FCMI [52] implemented fair clustering by mutual information theory, maximizing mutual information between nongroup information and cluster information and minimizing mutual information between group information and cluster information. Fair-MVC [55] explored the fairness issue on multi-view scenarios by enforcing the distribution of sensitive attributes within each cluster to be uniform. However, it excessively emphasizes fairness constraints but neglects the true cluster distribution structure of samples with sensitive attributes, leading to incorrect clustering assignments for them. As shown in Figure . 1(b) and (c), most fair clustering methods force the samples containing sensitive attributes in each cluster to be evenly distributed, while ignoring the clustering structure of the samples themselves.In mitigating the challenge, in this paper, we propose a deep fair multi-view clustering method named DFMVC, whose framework is shown in Figure . 1. We initially leverage deep autoencoder to extract view-specific features from different views. Then, to further exploit the discriminative and complementary information of the multi-view data, we transform the features from different views into semantic spaces and leverage contrastive learning to obtain consistent semantic representations. Finally, in order to achieve optimal fairness in clustering results while ensuring that sensitive attributes do not impact the clustering performance, we design a target cluster distribution-guided fairness learning module. In this module, we learn a target cluster distribution by fusing viewspecific representations. Additionally, we utilize a constraint based on KL-divergence to align the distribution of sensitive attributes with the target cluster distribution to achieve optimal fairness in clustering results. We summarize the main contributions of this paper as follows:  
related work 2.1 multi-view clustering: 
 In recent years, MVC [8, 38, 41, 42, 49] has gained considerable attention, with a growing body of research addressing this area.The current multi-view clustering method can be broadly classified into two main categories: traditional MVC methods and deep MVC methods. Traditional MVC methods can be divided into three main categories: (1) Non-negative matrix factorization-based methods [54] . It usually utilizes matrix factorization to decompose the integrated feature matrix, aiming to learn a shared representation. [45] introduced gradual factorization of multi-view data matrices into representational subspaces lay-by-layer and generated one clustering in each layer. (2) Subspace-based methods, aiming to learn consistent subspace representation across multiple views. For instance, [41] proposed a deep structured multi-pathway network for multi-view subspace clustering task. (3) Graph-based methods [32, 35] utilize multi-view data to construct graph structure. However, these traditional methods tend to learn shallow representations of multi-view data, thereby constraining the discriminative capability of the obtained representations.Benefiting from deep learning, deep neural networks have the power ability to extract feature representations. In recent years, many deep multi-view clustering methods have been proposed [18, 27, 29, 49] . First, one of the most representative deep MVC methods is based on deep autoencoders. These methods aim to learn consensus representation by minimizing the reconstruction loss by instances of multiple views. DMSC-UDL [40] employed deep autoencoder to obtain representations of different views, imposing different constraints to obtain self-expression layers. Second, contrastive learning-based MVC methods. By maximizing the distance between positive pairs and minimizing the distance between negative pairs, MVC can learn more discriminative clustering information. AGCL [44] explored intra-cluster consistency and inter-view consistency through within-view graph contrastive learning and cross-view graph consistency learning. Last, adversarial networkbased MVC methods employ adversarial training to further capture the data distribution. DAMC [25] utilized adversarial training to capture the data distribution and disentangle the latent space. These methods achieve better performance.Recently, there has been significant interest in the machine learning community regarding the topic of fairness in clustering. To mitigate or even eliminate the impact of sensitive attributes, many efforts have been devoted to fair clustering [1, 4, 7, 9] . Based on how fairness is incorporated, the existing works can be roughly divided into three categories: (1) Pre-processing method. It transforms the original data to satisfy fairness constraints and uses classic clustering algorithms to achieve fair clustering. For example, [10] first divided the original data into several chunks with fairness constraints (i.e., fairlets), and then used the classic clustering algorithm on these fairlets to obtain fair clusters. However, constructing the fairlets requires at least quadratic running time, which poses a significant challenge in practical applications. Therefore, ScFC [2] proposed a tree metric to construct fairlets in near-linear time. (2) In-processing methods. Different from pre-processing methods, it adds fairness as a constraint to the objective function for joint optimization. For instance, SpFC [20] treated fairness as a constraint and integrated it into spectral clustering. (3) Post-processing methods [3] . In contrast to pre-processing methods, post-processing methods techniques Table 1 : Basic notations used in the whole paper.The input data matrix of 𝑣-th view XThe reconstruction data matrix of 𝑣-th view Z 𝑣The embedded feature of 𝑣-th view H 𝑣The embedded feature of 𝑣-th view SThe sensitive feature Z 𝐺The global feature of all views V nThe view number of the multi-view dataset NThe number of samplesd vThe dimension of input data matrix d zThe dimension of embedded feature KThe number of cluster q 𝑖𝑘 Soft clustering assignment p 𝑖𝑘 Target assignment focus on converting the existing clustering result into a fair result through the resolution of a linear programming problem.Inspired by the achievement of deep clustering [12] [13] [14] 17] , deep fair clustering has attracted widespread attention. For instance, Towards [39] proposed a method to learn a fair embedding (i.e., fairoid) by enforcing cluster centroids to be equi-distant from each fairoid. DFC [22] explored fair clustering by adversarial training. FCMI [52] researched deep fair clustering by mutual information theory. However, these works mainly focus on a single view. Fair-MVC [55] delved into fair clustering on multi-view data, but this method limits the clustering performance by forcing the fraction of different groups in each cluster to be approximately identical to the entire dataset. 
methodology: 
 In this section, we introduce a novel Deep Fair Multi-view Clustering. The comprehensive DFMVC framework is illustrated in Figure 1 . DFMVC mainly consists of three key modules: Multi-view Data Reconstruction Module, Semantic Contrastive Learning Module, and Cluster Distribution-guide Fair Learning Module. Specific descriptions of these modules will be provided in the subsequent sections.Given a set of multi-view data X = {𝑋 1 , 𝑋2 , ..., 𝑋 𝑉 𝑛 , 𝑆 }, where 𝑋 𝑣 ∈ R 𝑁 ×𝑑 𝑣 (𝑣 = 1, 2, ..., 𝑉 𝑛 ) is the input data matrices for the 𝑣-th view.𝑆 ∈ R 𝑁 ×𝑑 𝑠 is the sensitive features (e.g., gender, race, etc.). 𝑉 𝑛 is the number of views, 𝑁 is the number of samples, 𝑑 𝑣 is the dimension of 𝑋 𝑣 , and 𝑑 𝑠 is the dimension of sensitive features. Assume that 𝐾 is the number of clusters. The samples with the same semantic labels can be grouped into the same cluster. Therefore, 𝑁 samples can be categorized into 𝐾 different clusters. A summary of essential notations is provided in Table 1 .Due to original multi-view data usually containing redundancy and random noise, we need to initially learn representative feature representations from the original data. In particular, the autoencoder [16, 33] stands out as a commonly employed unsupervised model capable of transforming original data features into a lowdimensional feature space. Specifically, for the 𝑣-th view, let 𝑓 𝑣 𝜃 𝑣 (•) (1 ≤ 𝑣 ≤ 𝑉 𝑛 ) denote the encoder nonlinear function. In the encoder, the network can learn the low-dimensional features as follows:EQUATIONwhere z 𝑣 𝑖 ∈ R 𝑑 𝑧 is the embedded feature representation in 𝑑 𝑧 dimensional feature space of 𝑖-th sample from the 𝑣-th view x 𝑣 𝑖 . Then, the decoder reconstructs the sample by the feature representation 𝑧 𝑣 𝑖 . Let 𝑔 𝑣 𝜙 𝑣 (•) (1 ≤ 𝑣 ≤ 𝑉 𝑛 ) represent the decoder function. In the decoder part, the reconstructed sample x𝑣 𝑖 is obtained by decoding z 𝑣 𝑖 :EQUATIONLet L 𝑟 be the reconstruction loss from input X 𝑣 ∈ R 𝑛×𝑑 𝑣 to output X𝑣 ∈ R 𝑛×𝑑 𝑣 , 𝑛 denote the number of samples in a batch. The reconstruction loss is formulated as:L 𝑟 = 𝑉 𝑛 ∑︁ 𝑣=1 L 𝑣 𝑟 = 𝑉 𝑛 ∑︁ 𝑣=1 X 𝑣 -X𝑣 2 2 = 𝑉 𝑛 ∑︁ 𝑣=1 𝑛 ∑︁ 𝑖=1 x 𝑣 𝑖 -𝑔 𝑣 𝜙 𝑣 (z 𝑣 𝑖 )In this subsection, we will introduce a semantic contrastive learning module to obtain consistent and discriminative features. Based on Z 𝑣 = 𝑓 (X 𝑣 ; 𝜃 𝑣 ), we can obtain embedded features. We stack two linear layers and a successive softmax function on embedded features to produce a cluster assignment, which is computed by:𝑓 𝑊 𝑣 : {Z 𝑣 } → {H 𝑣 } (1 ≤ 𝑣 ≤ 𝑉 𝑛 ), where 𝑊 𝑣 is the learnable parameters of linear layers. Inspired by recently proposed contrastive learning techniques [6] , we can leverage these methods on the embedded features to explore consistency across multiple views. By doing so, We can acquire cluster assignment probability matrices H 𝑣 ∈ R 𝑁 ×𝐾 (1 ≤ 𝑣 ≤ 𝑉 𝑛 ) for all views, where 𝐾 is the number of clusters. Let h 𝑣 𝑖 denote the 𝑖-th row in H 𝑣 , h 𝑣 𝑖 𝑗 represent the probability that instance 𝑖 belongs to cluster 𝑗 in the 𝑣-th view, and h 𝑣 𝑗 represents a cluster assignment of the same semantic cluster. The semantic label of instance 𝑖 is determined by the highest probability within h 𝑣 𝑖 . The instances should share the same semantic information. we define the similarity between two cluster assignments h 𝑣 1 𝑗 and h 𝑣 2of cluster 𝑗 asEQUATION)where ⟨•, •⟩ is dot product operator, 𝑣 1 and 𝑣 2 denote two different views. In our method, the cluster probabilities of instances among different views should be similar, as these instances collectively characterize the same sample. In addition, the instances in multiple views should be independent of each other, as they are utilized to characterize different samples. Therefore, for the 𝑣-th view, the same semantic labels h 𝑣 𝑗 should have (𝑉 𝑛 -1) positive cluster assignment pairs and 𝑉 𝑛 (𝐾 -1) negative cluster assignment pairs when considering h 𝑣 𝑗 and 𝐾 cluster across 𝑉 𝑛 views. Specifically, we consider instances of the same sample across different views as positive pairs, instances of different samples within the same view, as well as instances across different views, as negative pairs.The similarities among positive pairs should be maximized, and those among negative pairs should be minimized. The semantic contrastive loss between h 𝑣 1 𝑗 and h 𝑣 2 𝑗 is defined as follows:EQUATIONEQUATIONEQUATIONEQUATIONwhere 𝜏 𝐿 is a temperature parameter, (h𝑣 1 𝑗 , h 𝑣 2 𝑗) is a positive cluster assignment pair between 𝑣 1 -th view and 𝑣 2 -th view, (h 𝑣 1 𝑗 , h 𝑣 1 𝑘 ) is a negative cluster assignment pair between 𝑗-th cluster and 𝑘-th cluster in the 𝑣 1 view, and(h 𝑣 1 𝑗 , h 𝑣 2 𝑘) is also a negative cluster assignment pair between 𝑣 1 -th view and 𝑣 2 -th view. Therefore, semantic contrastive loss included across multiple views could be defined as:EQUATION)where𝑟 𝑣 𝑗 = 1 𝑁 𝑁 𝑖=1 ℎ 𝑣 𝑖 𝑗 .The first part of Eq. ( 6) aims to learn the clustering consistency for all views. It pulls pairs of cluster assignments from the same cluster closer together and pushes cluster assignments from The second part of Eq. ( 6) is a regularization term, which is used to prevent all samples from being assigned exclusively to a single cluster. Specifically, if ℎ 𝑣 𝑖 𝑗 = 1 for all 𝑖 = 1, 2, ..., 𝑁 , it means that all instances only belong to 𝑗-th cluster, at this time,𝑟 𝑣 𝑗 is 1, therefore, 𝑟 𝑣 𝑗 log 𝑟 𝑣 𝑗 is 0. When 0 ≤ ℎ 𝑣 𝑖 𝑗 ≤ 1, at this time, 𝑟 𝑣𝑗 log 𝑟 𝑣 𝑗 ≤ 0. By this regularization term, we can make sure that each cluster has at least one sample. Hence, this loss can encourage the acquisition of clustering information that is both discriminative and consistent across multiple views.In this subsection, we will introduce a cluster distribution-guide fair learning module to accomplish fair clustering. Inspired by other deep clustering [22, 46] , we employ target cluster distribution 𝑃 to guide the learning of sample distributions with sensitive attributes. Specifically, we design a weighted adaptive fusion mechanism, aiming to fuse the embedded features from each view into a unified embedded feature. We will learn 𝑉 𝑛 embedded feature for each Random select a minibatch of samples;4:Computing Z 𝑣 and Z 𝐺 by Eqs. ( 1) and ( 7), respectively;5:Computing 𝑞 𝑖𝑘 and 𝑝 𝑖𝑘 by Eqs. ( 8) and ( 9), respectively;6:Computing L 𝑟 , L 𝑐 , L 𝑓 by Eqs. ( 3), ( 6) and (10) 7:Optimizer parameters by Eqs. ( 11); 8: end for 9: Calculate the semantic labels by Eqs. (12) . view data, i.e., {Z 1 , Z 2 , ..., Z 𝑉 𝑛 }, where Z v = 𝑓 (X v , 𝜃 𝑣 ). We get a common representation Z based on all embedded features. It can be calculated as:Z = 𝑉 𝑛 𝑣=1 𝑎 𝑣 Z 𝑣 𝑉 𝑛 𝑣=1 𝑎 𝑣 (7)where𝑎 𝑣 ∈ 𝐴, 𝐴 = [𝑎 1 , 𝑎 2 , ..., 𝑎 𝑉 𝑛 ], Z 𝑣 ∈ R 𝑛×𝑑 𝑧 , Z ∈ R 𝑛×𝑑 𝑧 , 𝑛denote the number of samples in a batch. Z represents the global embedded feature obtained after fusing each view. For example, if we have two views, it can write thatZ = 𝑎 1 Z 1 +𝑎 2 Z 2 𝑎 1 +𝑎 2 .𝐴 is a set of learnable parameters, and it will learn an appropriate weight for each view. If a view contains more information, the greater its corresponding weight. Then, we can define soft assignment 𝑞 as follows:𝑞 𝑖𝑘 = (1 + ||z 𝑖 -𝑐 𝑘 ||/𝛼) -𝛼 +1 2 𝑘 ′ ∈ [𝐾 ] (1 + ||z 𝑖 -𝑐 𝑘 ′ ||/𝛼) -𝛼 +1 2 (8)where 𝑐 𝑘 represents the cluster centroid, 𝛼 is the degree of freedom of Student's t-distribution. In our experiment, we set 𝛼 to 1. 𝑞 𝑖𝑘 can be interpreted as the probability the 𝑖-th sample is assigned to the 𝑘-th cluster.We follow [46] to use Student t-distribution for assignment, but the difference is that performing this operation on each protected subgroup to prevent any single cluster from being dominated by samples with the same sensitive attribute. The auxiliary target distribution 𝑃 is calculated by:EQUATIONwhere 𝑋 𝑔 is the subgroup that the instance belongs to. For example, if the sensitive feature is gender, it has two subgroups (males and females). Through this process, we can obtain target cluster distribution. The cluster distribution-guide fair loss is KL divergence between soft assignment and auxiliary target distribution:L 𝑓 = 𝐾𝐿(𝑃 || 𝑄) = ∑︁ 𝑠 ∈ [𝑆 ] ∑︁ 𝑥 𝑣 ∈𝑋 𝑔 ∑︁ 𝑘 ∈ [𝐾 ]𝑝 𝑖𝑘 log 𝑝 𝑖𝑘 𝑞 𝑖𝑘 (10) By minimizing this loss, making 𝑞 𝑖𝑘 gradually approach the cluster target distribution 𝑝 𝑖𝑘 . On the one hand, it can learn more discriminative clustering information, on the other hand, it also can learn fair representations. The object function of the proposed DFMVC contains the reconstruction loss L 𝑟 , the semantic contrastive learning loss L 𝑐 , and the cluster distribution-guide fair learning loss L 𝑓 . In summary, the object of DFMVC is formulated as follows:EQUATIONwhere 𝛼, 𝛽 are the trade-off parameters. The detailed learning process of our DFMVC is shown in Algorithm 1.The proposed method aims to learn feature consistency and obtain semantic labels across multiple views. Let ℎ 𝑣 𝑖 be the 𝑖-th row of 𝐻 𝑣 , and let ℎ 𝑣 𝑖 𝑗 represent the 𝑗-th element of ℎ 𝑣 𝑖 . Specifically, ℎ 𝑣 𝑖 is 𝐾-dimensional soft assignment probability, where 𝑁 𝑖 ℎ 𝑣 𝑖 𝑗 = 1. When the training process of the network is completed, the semantic label can be predicted by:EQUATION 
experiment: 
 In this section, we conduct the experiments to verify the effectiveness of the proposed DFMVC by answering the following questions:• RQ1: How effective is DFMVC for deep fair clustering?• RQ2: How does the proposed module influence the performance of DFMVC? • RQ3: What is the clustering structure revealed by DFMVC? • RQ4: How about the convergence about DFMVC? • RQ5: How do the hyper-parameters impact the performance of DFMVC?Fairness Datasets We conduct extensive experiments to verify the effectiveness of DFMVC on four fairness datasets, including Credit Card dataset [51] , Zafar dataset [51] , Bank Marketing dataset [51] , and Law school dataset [21] . Specifically, the Credit Card dataset describes the customers' default payments in Taiwan and this dataset consists of 5000 samples with 24 attributes. The sensitive feature is gender. The Bank Marketing dataset is associated with direct marketing campaigns of a Portuguese banking institution and it aims to see if the product (bank term deposit) would be('yes') or not ('no') subscribed. The sensitive feature in this dataset is marital status. This data set consists of 5000 instances and 17 attributes.The Law School dataset is to predict whether a candidate would pass the bar exam, and it consists of 3600 samples with 12 features. The dataset's sensitive feature is gender. Last, the Zafar dataset is a widely used synthetic dataset, where one binary value is generated 2 .Evaluation Metrics To evaluate the effectiveness and superiority of our DFMVC, we adopt the widely used metrics, i.e., normalized mutual information (NMI) and balance (BAL) [10, 43] . NMI is used to evaluate the clustering performance of our model, while BAL is used to assess the fairness of our model. Balance is defined as follows:EQUATIONWhere 𝐶 𝑖 represents the 𝑖-th cluster, 𝑠 𝑗 represents 𝑗-th protected subgroup. For example, for gender, there are two protected groups, namely males and females. Typically, the upper limit of balance is determined by the distribution of the sensitive feature, with a higher balance value suggesting a fairer result.The experiments are executed using the following hardware configuration: Intel Core i7-9700K CPU, NVIDIA GeForce RTX 2080Ti GPU, and 64GB RAM. Additionally, all experiments make use of the Pytorch platform. In the case of DFMVC, we utilize the Adam [19] optimizer to minimize the total loss.The proposed DFMVC is compared with the following baselines:• K-means [31] : a method is employed to divide samples into multiple clusters, with each sample assigned to the cluster closest to it. • DEC [46] : a deep learning approach is utilized to simultaneously learn feature representations and cluster assignments using neural networks. • CC [23] : a clustering method based on contrastive learning, which simultaneously optimizes instance-level and clusterlevel contrastive loss. • MvDSCN [57] : a multi-view deep subspace clustering network is employed to learn a self-representation matrix from multiple views.• DCP [28] : a multi-view clustering approach that utilizes contrastive learning to ensure consistent representations and fills in missing views using dual prediction modules.• APADC [47] : a multi-view clustering method based on subspace learning is employed to align view distributions by minimizing disparity loss.• MFLVC [49] : a multi-level feature learning for contrastive multi-view clustering, learns different levels of features. • Fair-MVC [55] : a method that merges fair clustering with multi-view clustering, enhancing feature representation through the addition of contrast regularization.To minimize the impact of randomness, each method runs many times, and the outcomes are presented as mean values accompanied by their respective standard deviations. For the Incomplete Multi-view Clustering method, we set the missing rate to 0. In the case of our DFMVC, we adopt a two-stage training approach. We utilized the autoencoder as the pre-training model, and reconstruction loss as the loss function for parameter initialization. We assign 200 epochs for pre-training and 200 epochs for the subsequent training process. In our proposed DFMVC, a batch size of 100 is employed consistently across all datasets. The learning rate is set at 0.0001.In this subsection, we conduct experiments to demonstrate the superiority of DFMVC with 8 baselines on four fairness datasets. In Table 3 , the NMI and BAL of all the compared methods on the four datasets are presented. From these results, we have the following observations. 1) compared to classical deep multi-view clustering algorithms, our approach, DFMVC, consistently yields superior clustering results across the majority of datasets. Using the Credit Card dataset as an example, DFMVC outperforms its nearest competitors by significant margins of 8.4% and 2.48% in terms of NMI and BAL, respectively. We conjecture the reason is that the majority of methods fail to consider fairness and the learned representations are not as discriminative as desired. 2). Thanks to the consideration of fairness and the adoption of contrastive learning strategies to learn feature consistency among multiple views and obtain semantic information by contrasting clustering assignments among different views, our method can perform better. To verify the effectiveness of the semantic contrastive learning mechanism and the cluster distribution-guide fair learning module, we conduct experiments on all datasets. For simplicity, we adopt "(w/o) L_C" and "(w/o) L_F" to denote the reduced models by removing the semantic contrastive learning module and cluster distribution-guide fair learning module respectively. As shown in Table 4 , the best performance can be achieved when all loss terms are considered. Take the Credit Card dataset as an example, we can observe that the clustering performance will decrease without any of our proposed modules, indicating that each module contributes to boosting the performance.In this subsection, we perform visualization experiments to intuitively showcase the effectiveness of DFMVC. Specifically, we employ t-SNE algorithm [37] to visualize the distribution of raw features and learned embeddings of DFMVC on the Credit Card and the Zafar dataset. Take the Zafar dataset as an example, the Zafar dataset is a widely-used synthetic dataset, and we generate a binary value as its sensitive feature. Blue dots represent binary 0, and red dots represent binary 1. From the Figure . 3(c), we can observe an even distribution of binary 0 and binary 1 in raw features. In Figure . 3(d), the binary values are evenly distributed in each cluster. This result suggests that our method demonstrates independence from sensitive attributes. That is, our method fulfills the objective of fair clustering.In this subsection, we explore the convergence of the DFMVC method. Specifically, we explored pre-training and formal training separately. To validate the convergence of the DFMVC method, we compute the results of the loss function by Eqs. ( 11) and (3) during training. As shown in Figure 4 , it depicts the curves of the loss function results across all the datasets. The value of the loss function defined in Eq. (3) exhibits a sharp decrease in the initial iterations, followed by a gradual decline until convergence is achieved. Similarly, we observe a comparable pattern in the variations of the loss function values defined in Eq. (11) . These observations underscore the effectiveness of the convergence property inherent in the DFMVC method. We perform experiments on two representative datasets, namely the law school and bank marketing datasets, to access the impact of the 𝛼 and 𝛽 parameters in the proposed DFMVC method. The 𝛼 and 𝛽 are chosen from {0.01, 0.1, 1, 10, 100} for DFMVC. As can be observed in Figures 5 and 6 , we can find that clustering performance and fairness achieved by the DFMVC in terms of the NMI and BAL values obtained with different combinations of 𝛼 and 𝛽. From the figures, we can observe that different parameter combinations have a slight impact on the clustering performance but do not have much impact on the balance value. This shows that our method is not so sensitive to the selection of hyper-parameters, indicating the stability of our method. 
conclusion: 
 In this paper, we propose a DFMVC method that learns more discriminative and fairer representations for MVC. Specifically, we use AutoEncoder to initialize the parameters. Then, we fuse the representations extracted from each view, and by the cluster distributionguide fair learning module, learn fairer representations. To further use the diversity of multi-view data, we use contrastive learning to obtain feature consistency and semantic labels. We conduct extensive experiments and ablation studies on four datasets to validate the superiority of the model and the effectiveness of each component in our method.  
 
 Appendix: 2 (3)During the pre-training stage, we utilize the reconstruction loss function for training, aiming to initialize the parameters. Each encoder and decoder consists of four layers, and the nonlinear rectified linear unit (ReLU) function is chosen as the activation function in the deep autoencoder. 


Decision: The existing fair multi-view clustering methods are built on the uniform distribution of attributes. In contrast, this paper constructs a fairness constraint by using the distribution of clusters, which can preserve the consistent and discriminative representation across different views. They used the contrastive loss on semantic features and alignment loss on the distribution of sensitive attributes. Many experiments show the effectiveness of the proposed method against different MVC algorithms. 

We have five reviewers who carefully checked this paper. They raise many concerns about the technical and experimental details as well as the fairness description. After rebuttal, all the reviewers are satisfied with the current form of this paper. All of them voted to accept this paper. Considering the good presentation, structure, and experimental results, this paper deserves to be accepted to ACM MM 2024.

Reviews: review: paper_summary: paper_summary: The main idea of this paper is to explore the fairness in deep multi-view clustering. The authors proposed Deep Fair Multi-View Clustering (DFMVC) method that learns a consistent and discriminative representation instructed by a fairness constraint constructed from the distribution of clusters. It incorporates contrastive constraints on semantic features from different views to obtain consistent and discriminative representations for each view. The proposed method aligns the distribution of sensitive attributes with the target cluster distribution to achieve optimal fairness in clustering results. Experimental results on the multi-view datasets that contains sensitive attributes demonstrate the validity of the proposed deep fair multi-view clustering method and its superior advantages over several state-of-the-art algorithms.
strength_weakness: strength_weakness: (1) The idea of this paper is innovative, and the main advantages of the proposed method are designing a novel deep fair multi-view clustering method. The proposed method explores the unfairness problem of multi-view clustering. The cluster distribution is utilized to guide the distribution of sensitive attributes, which alleviates the problem that samples with certain sensitive attributes are incorrectly clustered. 
(2) The effectiveness of the method is somehow obvious. Contrastive learning is a popular method to learn better features, with which the clustering result can be improved. For fairness, aligning distribution via KL divergence is consistent with the conceptual definition of fairness in machine learning. Thus, the work’s effectiveness holds in theory.
(3) Extensive experiments on four datasets are conducted, and the performance in terms of fairness and clustering is evaluated with the established metrics.
limitations: limitations: (1) The literature review seems not enough, since there are many works on deep multi-view clustering. More should be involved and analyzed, or the motivations of the work are not convincing.
(2) The authors only compare the proposed method with one fair multi-view clustering method (Fair-MVC). Although there are few fair multi-view clustering methods, fair clustering methods in single views have been widely studied, resulting in many related works proposed. Therefore, to better demonstrate the superiority of the work, more single-view fair methods are recommended to be compared.
(3) It is noted that the paper needs careful editing by someone with expertise in technical English editing paying particular attention to English grammar, spelling, and sentence structure so that the goals and results of the study are clear to the readers.
paper_summary: paper_summary: The paper proposes a new deep fair multi-view clustering method called DFMVC. Mining inter-view consistency information and preserving discriminative information of each view based on contrastive learning and reconstruction loss. Divide subgroups by sensitive attributes and generate target soft distributions within the subgroups to guide clustering optimization to achieve fair clustering.
strength_weakness: strength_weakness: 1.	The paper is clearly structured and easy to read.
2.	The idea is very simple and novel.
3.	Extensive experiments on multiple datasets confirm the effectiveness of the proposed method.
limitations: limitations: 1.	The description in Figure 1 is not very clear. What does a sample without sensitive attributes refer to?
2.	There are some errors. For example, in the last paragraph of the introduction section, the framework diagram should be Figure 2 instead of Figure 1.
3.	hj in Equation 4 is the j-th column of the H matrix, and the H matrix is N*K, which indicates that Equation 4 uses all samples at the same time to calculate similarity. Since the training process is based on min-batch, please explain how this is implemented. If you are calculating the similarity within min-batch, since Equation 4 is batch size sensitive, additional batch size ablation experiments are required.
4.	Why does the denominator in Equation 5 have -e^1/τ, and is there a lack of positive terms in the denominator?
5.	αv in Equation 7 is the same for all samples of view v, which assumes that all samples in the same view have the same importance.
6.	The subscripts of Σ in Equation 10 are confusing.
paper_summary: paper_summary: The paper is concentrated on the deep fair multi-view clustering. The authors first introduce the unfairness problem confronted by existing multi-view clustering, as illustrated by an example in Figure 1. Then, a distribution alignment based solution is developed, which leverages target cluster distribution to guide fair clustering. To simultaneously achieve high clustering performance, a contrastive learning module is introduced to obtain the consistent and discriminative representation. The improved discriminativeness helps to better sort samples into different clusters. In this way, the method achieves both fairness and clustering results. To support the conclusion of the paper, the authors also conduct a number of experiments, whose results indicate the superiority of the proposed method. In summary, the paper is technically sound, and the conclusion are well supported by the theoretical analysis and experimental results.
strength_weakness: strength_weakness: (1) An innovative deep multi-view clustering model is developed. It is interesting and novel to leverage target cluster distribution to guide fair clustering in deep multi-view clustering, which is a practical scenario in real-world applications, for the multi-view clustering task. Figure 1 also clearly shows the research motivations.
(2) The technique is sound and useful for the multi-view clustering tasks. First, it employs contrastive learning to enhance the discriminative features extracted. Second, it leverages target cluster distribution to guide fair clustering, leading to that the clustering results are independent of sensitive attributes.
(3) Extensive comparison experiments on four real world datasets with sensitive attributes have been conducted to prove the effectiveness of the proposed model. Compared to the state-of-the-art methods, the proposed deep fair multi-view clustering method achieves promising clustering performance as well as fairness.
limitations: limitations: (1)	For the writings, there are several typos to be corrected. Some symbols are not well defined. The authors should carefully check the entire paper and revise it carefully.
(2)	For the experiments, the authors describe each dataset from the perspective of sensitive attributes. However, the different views of these datasets are less introduced. The authors should explain each view of the four datasets adopted by the experiments.
(3)	Some details require more introduction. For example, during the adaptive fusion, how to initialize $A$ is not well illustrated. For another, how to obtain the final clustering result?
paper_summary: paper_summary: This paper presents a novel deep fair multi-view clustering method, which attempts to learn a consistent and discriminative representation for clustering guided by a fairness constraint constructed from clustering distribution. It aims to address the unfairness issue existing in current multi-view clustering methods, which is a significant research topic. Its main theoretical contributions include: (1) It incorporates a contrastive constraint to learn a consistent representations of multi-view data, which can better obtain the discriminative information to improve clustering performance; (2) By aligning the distribution of sensitive features with the target cluster distribution, it well ensures fairness and meanwhile maintains the satisfactory clustering results. Besides, extensive analyses and experiments are performed, whose results well demonstrate the method’s superiority.
strength_weakness: strength_weakness: 1.	The research topic is significant, since fairness attracts more and more attention, while this issue still lacks exploration in multi-view clustering. The paper is relatively well-written with reasonable organization and clear motivations. The literature review covers most related works in terms of multi-view clustering and fairness and provides a deep analysis of them.
2.	The work is technically sound, which employs contrastive learning to enhance the discriminativeness of the consistent feature extracted from different views, and by introducing a fairness constraint constructed from the sensitive attribute distribution in different clusters, it well ensures the fairness. The method design is reasonable. Sufficient experiments are conducted, and by comparing the proposed method with existing multi-view clustering algorithms, its effectiveness and superiority in both multi-view clustering performance and fairness is well demonstrated.
limitations: limitations: 1.	Although this paper is overall well-written, there still exist some typos and grammar mistakes. Some details still need to be amended. For example, the notation $Z^G$ in Table 1 is not clearly defined or introduced, and maybe it should be a mistake and should be revised as $Z$.
2.	Figure 2 should be further improved. The symbol $D_2$ and $E_2$ in it seem to be the decoder and encoder for the $n$th view, and thus they should be $D_n$ and $E_n$.
3.	In Figure 3, the authors use different colors to represent different sensitive attributes, which can effectively demonstrate the fairness of the proposed method. However, the selected two colors should be changed because these two colors make readers not able to well distinguish them.
paper_summary: paper_summary: The paper proposes a novel fair multi-view clustering method by leveraging contrastive learning to extract consistent and discriminative features from different views, thus distinguishing multi-view data better. The work also considers the fairness issue in multi-view clustering, which is ignored by most related works and has gained widespread attention in academia, which aims to alleviate the model bias on a certain sensitive attribute set. The authors proposed to achieve the optimal fairness in multi-view clustering tasks by aligning the distribution of sensitive features with the target cluster distribution. With the carefully designed experiments on several datasets as well as the comparison with existing works, the superior clustering performance and the model fairness are well demonstrated.
strength_weakness: strength_weakness: The paper follows a well-structured organization. The strengths of the paper include:
(1) Fairness has received increasing attention. Despite there are some works exploring the fairness issue in multi-view learning tasks, fairness problem in multi-view clustering still remains underexplored. The work focuses on a significant research issue.
(2) The idea of the work seems technically sound. By utilizing the target cluster distribution to guide fair clustering, the unfairness problem in multi-view clustering can be effectively alleviated. The contrastive learning module helps to enforce the learned multi-view feature to be more discriminative, thus improving the clustering performance. The design is reasonable.
(3) Extensive experiments demonstrate the effectiveness of the work in achieving fairness and improving clustering performance. Comparison with existing works.
limitations: limitations: There are several limitations, summarized as follows:
(1)	In Table 1, both $H^v$ and $Z^v$ represent the embedded feature of the $v$-th view, what is the difference? Are the two symbols share the same meaning? If not, the authors should add more description to distinguish them.
(2)	The fairness of the proposed method is evaluated via experiments on several fairness datasets. However, the method in this work to achieve fairness, i.e., by aligning the distribution of sensitive attributes with the cluster distribution, should be further discussed by providing more justifications.
(3)	The paper should be proofread to improve the usage of English. Please go through this paper carefully, and correct all the typos.