# Entropy-Based Multiview Data Clustering Analysis in the Era of Industry 4.0

Authors: Yi Gu; Kang Li
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

In the era of Industry 4.0, single-view clustering algorithm is difficult to play a role in the face of complex data, i.e., multiview data. In recent years, an extension of the traditional single-view clustering is multiview clustering technology, which is becoming more and more popular. Although the multiview clustering algorithm has better effectiveness than the single-view clustering algorithm, almost all the current multiview clustering algorithms usually have two weaknesses as follows. (1) The current multiview collaborative clustering strategy lacks theoretical support. (2) The weight of each view is averaged. To solve the above-mentioned problems, we used the Havrda-Charvat entropy and fuzzy index to construct a new collaborative multiview fuzzy c-means clustering algorithm using fuzzy weighting called Co-MVFCM. The corresponding results show that the Co-MVFCM has the best clustering performance among all the comparison clustering algorithms.


---

## Body

## 1. Introduction

In the era of Industry 4.0, as the methods of data collection become more and more diverse, the complexity of data is also increasing. For example, a driverless car will collect environmental data through a variety of sensors and conduct analysis and processing from multiple views while driving. In unsupervised learning, clustering is usually used for complexity data analysis. However, traditional clustering methods, such as K-means [1, 2], fuzzy C-means (FCM) [3, 4], maximum entropy clustering (MEC) [5, 6], and possibilistic C-means (PCM) [7, 8], are all designed for single-view data analysis. When the single-view algorithms [9–11] encounter a multiview clustering task, the common practice is to first consider each view independently, treating each view as an independent clustering task. After finishing each clustering task of each view, the integrated learning mechanism [12] is used to select an appropriate integrated learning strategy to integrate multiple clustering results and then get the final clustering results. However, due to obvious deviation of clustering results in a certain view or great difference of clustering results among different views, the multiview strategy which artificially separates each view for independent analysis may result in inaccurate global partitioning results obtained by integrated learning or unstable algorithm performance.In many real applications, multiview representation of data is becoming more and more popular [13], especially in the field of medicine [14, 15]. For example, people’s living standard and economic situation have been improved since entering the twentieth century. But the incidence rate of cancer has increased by nearly 50% compared with that of the 1880s due to environmental pollution, food safety, and working pressure. However, with the continuous improvement of medical level, various detection methods such as laboratory examination (routine examination, serological examination, gene or gene product examination, etc.), imaging and endoscopy (X-ray examination, B-ultrasound examination, CT examination, radionuclide imaging, etc.), and cytopathological examination (puncture biopsy, forceps biopsy, section analysis, etc.) have been proposed and applied. These methods can be used to analyze the suspected patients from different views. This is a typical multiview data representation problem. The above examples reveal that developing various multiview clustering algorithms is very necessary for us to better observe and mine the essence of data from the viewpoint of its diverse descriptions and accordingly obtain a better clustering result that simultaneously satisfies every representation (view). Introducing the multiview technology into the traditional clustering analysis method so that there is collaborative learning in the clustering process is considered to be an effective solution. In recent years, some effective multiview clustering methods have been proposed using the above strategies. Yamanishi et al. [16] proposed a collaborative clustering algorithm Co-EM algorithm that can be used to solve multiview problems based on the EM algorithm from the perspective of probability and test the proposed algorithm’s effectiveness through some text-like samples. Inspired by FCM algorithm, Pedrycz [17] controlled the fuzzy partition between the various views, constructed a divisional cooperative control function, and finally obtained the Co-FC algorithm. The algorithm has shown certain advantages on various datasets. More early related multiview clustering studies can be found in [18, 19].As mentioned above, a lot of researches have begun to focus on the construction of multiview clustering algorithm. Through the summary of the current research on multiview clustering, we find that the current research mainly focuses on the following aspects: (1) the early multiview clustering algorithm usually preprocesses the data itself, and the most direct method is to synthesize a multiview data into a single-view data through feature fusion and then use the data clustering analysis; (2) most of the multiview clustering algorithms proposed in recent years use collaborative learning strategy, which can enhance the performance of each view data in the process of clustering; (3) when most multiview clustering algorithms with collaborative learning ability treat each view, their common practice is to average the weight of each view. In particular, we take the most classic Co-FKM algorithm [20] as an example, which is one of the representative multiview clustering algorithms in recent years. The algorithm designs a very effective multiperspective collaborative learning strategy, which can make the data between different perspectives use membership to complete collaborative learning in the process of clustering. But the algorithm also has a fatal disadvantage, that is, it just treats each perspective equally and does not give each perspective different weights. In addition, the multiperspective collaborative learning strategy proposed by the algorithm also lacks the necessary physical meaning, which cannot explain why this collaborative learning strategy can contribute to the final clustering performance. In response to the above-mentioned challenges, in this study, a new view space division criterion is first constructed based on the Havrda-Charvat entropy, which is used to control the space division results across different views so that the space division results of each view tend to be as consistent as possible in order to obtain a more stable and more comprehensive global spatial division result. Furthermore, we introduced fuzzy index and fuzzy weights to adaptively weight each view and effectively adjust the weight of each view during the clustering process so that the view with the clearest spatial division has a larger weight. Finally, a new collaborative multiview fuzzy c-means clustering algorithm using fuzzy weighting called Co-MVFCM is proposed by combining with the Havrda-Charvat entropy and fuzzy index. We summarized the contributions of this study here:
(1)
We construct a new view space division criterion using the Havrda-Charvat entropy. The built criterion can be used to control the space division results across different views(2)
We construct a view weighting mechanism using fuzzy index. The new view weighting mechanism can be used to recognize the importance degree of each viewOverall, our proposed Co-MVFCM algorithm not only has good space division ability but also has the ability to adaptively recognize the best view.


## 2. Related Work

When multiview clustering task is coming, Cleuziou et al. [20] proposed the Co-FKM method based on classical FCM. In Co-FKM, multiview clustering is achieved by a constraint of fuzzy membership degree which is aimed at keeping the partition result of each view as consistent as possible. Here, the Co-FKM method is defined as
(1)J=∑r=1R∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΔr,(2)Δr=λ1R−1∑r′=1,Rr′≠r∑i=1C∑j=1Nμij,r′m−μij,rmxj,r−vi,r2s.t.∑i=1Cμij,r=1,and μij,r∈0,1, 1≤j≤N, 1≤r≤R.By substituting Equation (2) to Equation (1), Equation (1) is simplified as
(3)J=∑r=1R∑i=1C∑j=1Nμ~ij,r,ηxj,r−vi,r2,where μ~ij,r,λ=1−λμij,rm+λ/R−1∑r′=1,r′≠rRμij,r′m, λ can be used to control the contribution of Δr, and μ~ij,r,λ is the average fuzzy membership degree of μij,rm and μij,r′m.The objective function of Co-FKM can be optimized by introducing Lagrange multipliers. So the fuzzy membership degreeμij,r and center vi,r are obtained as
(4)vi,r=∑j=1Nμ~ij,r,λxj,r∑j=1Nμ~ij,r,λ,μij,r=1∑h=1C1−λdij,r2+λ/R−1∑r′=1,r′≠rRdij,r′2/1−λdhj,r2+λ/R−1∑r′=1,r′≠rRdhj,r′21/m−1.To obtain a fuzzy division standard with global considerations, the fuzzy membership of each view can be computed by using the geometric mean method [20]. The specific expression is as follows:
(5)μ^ij=R∏r∈Rμij,r.From the Co-FKM algorithm, we can draw a general framework to represent multiview clustering, which is illustrated in Figure1. The Co-FKM algorithm incorporates the spatial division and approximation criteria across different views in the clustering and realizes collaborative learning across different views; it has more effective multiview clustering performance compared with traditional single-view integrated clustering technology. However, as we stated before, it still has challenges to be further solved.Figure 1
The principle of Co-FKM algorithm.


## 3. Collaborative Multiview Fuzzy Clustering (Co-MVFCM) Using Entropy Technology

In view of the two shortcomings of the current multiview clustering methods, the following two new technologies based on the entropy theory were introduced.(1)
We use the Havrda-Charvat entropy to construct a new view space division approximation criterion and find the maximum similarity component between each view so that while improving the performance of clustering, it also gives the view space division approximation criterion new physical meaning from the perspective of entropy(2)
We propose an entropy-weighted multiview clustering technology. By weighting each view, we can find the best view in the iterative optimization process and get the best fuzzy division result at the same time, in order to effectively control the weightFigure2 illustrates the new framework of multiview clustering.Figure 2
The principle of our proposed Co-MVFCM method.


### 3.1. Approximation Criterion of Space Division from Different Views Based on the Havrda-Charvat Entropy

In this study, the Havrda-Charvat entropy ofxj,r is defined as
(6)Ψmxj,r=11−21−m∑i=1Cμij,rm−1.It is obvious that if the fuzzy membership degreeUr=μij,rm is considered as a probability matrix, when the constraint ∑i=1Cμij,r=1 holds, Ψmxj,r is equal to 0. It is very intuitive to show that the uncertainty of xj,r belonging to each division in the sample set Xr of this view reaches the minimum value. That is to say, when the objective function reaches its minimum value, the Havrda-Charvat entropy of μij,r also reaches its minimum value.Although Equation (6) can ensure that the uncertainty of division can be minimized, it is limited to a single view. In order to expand it into a field of multiple views, in this study, we expand Equation (6) into the following expression form by referring to the relevant strategies used in [20]:
(7)Φ=11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1R−1∑i=1C∑j=1N∑r′=1,r′≠rRμij,r′m−1−μij,rm−1.So we have(8)Φ=1−β11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1K−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1.We observe from Equations (7) and (8) that η can be used to effectively regulate the weight relationship between the current view and the membership degree division of other views (0<β<1 and β=R−1/R). So we can get the weighted average of membership degree and finally make the membership degree division of each view as consistent as possible, so as to obtain the spatial division result with a more global view.


### 3.2. Multiview Adaptive Weighting Based on Fuzzy Index

In this study, we develop an automatic view weighting strategy using fuzzy index to recognize the best view. Supposewr represents the weight of view r under the condition that ∑r=1Rwr=1 and wr≥0, then wr can be considered as the probability distribution which is defined as
(9)fwr=∑r=1Rwrp.Fuzzy index technology is introduced through the above methods to make the objective function achieve the optimal entropy as much as possible, which is also the classical fuzzy c-means clustering principle [4].


### 3.3. Co-MVFCM

According to the above definitions, we propose our new multiview clustering method. The objective function of Co-MVFCM is(10)JCo−MVFCM=∑r=1Rwrp∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr,Φr=1−β11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1R−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1s.t.μij,r∈0,1,∑i=1Cμij,r=1 and ∑r=1Rwr=1, 1≤r≤R, 1≤j≤N.Obviously, the objective function contains two main parts. The first one isℓU,V,X=∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr which is derived from Havrda-Charvat and used for collaborative clustering. The essence of the first part is to find out as many similar parts among different perspectives as possible through multiview clustering technology and finally make the spatial division results of different views tend to be the same. The second part is ℓ~U,V,X,W=∑r=1RwrpℓU,V,X which is derived from fuzzy index. This part can be used to adaptively calculate the weight values of each view, and finally, when the algorithm reaches the optimal level, the optimal view partitioning results can be obtained according to the weight matrix of the views. The parameter β can be set to β=R−1/R. The parameter λ can be determined by using grid optimization [13, 21].To obtain the final result of space division with global characteristics, the integration strategy of global space division mentioned in [20] is abandoned in this study. We define a new integration strategy to obtain the final space division as
(11)U~=∑r=1RwrUrR.


#### 3.3.1. Optimization

The proposed multiview can be optimized by introducing Lagrange multipliers. In this section, we give three theorems to obtain updating rules in terms of fuzzy membership degree, view weights, and cluster centers.Theorem 1.
WhenUr and Wr are fixed, the cluster center can be solved by
(12)vi,r=∑j=1Nμij,rmxj,r∑j=1Nμij,rm.Proof.
By setting∂J/∂Vr=0, we have vi,r=∑j=1Nμij,rmxj,r/∑j=1Nμij,rm. Therefore, Theorem 1 is achieved.Theorem 2.
When the cluster centerVr and view weight matrix Wr are fixed, the fuzzy membership degree matrix can be solved by
(13)μij,r=1∑h=1C1−β/1−21−mλ+xj,r−vi,r2/1−β/1−21−mλ+xj,r−vh,r21/m−1.Proof.
By introducing Lagrange multipliersαj,r and considering the constraint ∑i=1Cμij,r=1, we have the following objective function:
(14)JCo−MVFCMμij,r,βj,r=−∑r=1R∑j=1Nοj,r∑i=1Cμij,r−1+∑r=1Rwrp∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr,where Φr=β1/1−21−m1/R−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1+1−β1/1−21−m∑i=1C∑j=1Nμij,rm−1.
By setting the partial derivative ofJCo−MVFCMμij,r,οj,r w.r.t. μij,r,οj,r to 0, i.e., ∂JCo−MVFCM/∂μij,r=0, we have
(15)μij,r=οj,rmwrp1−β/1−21−mλ+xj,r−vi,r21/m−1.
Similarly, with∂JCo−MVFCM/∂οj,r=0, we have
(16)∑i=1Cμij,r=1.
By combining Equations (15) and (16) to remove οj,r, we have
(17)μij,r=1∑h=1Cxj,r−vi,r2+1−β/1−21−mλ/xj,r−vh,r2+1−β/1−21−mλ1/m−1.
Therefore, the proof of Theorem2 is achieved.Theorem 3.
When the center matrixVr and the fuzzy membership degree matrix Ur are fixed, the weight matrix can be solved by
(18)wr=1∑r′=1R∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr/∑i=1C∑j=1Nμij,r′mxj,r′−vi,r′2+λΦr′1/p−1.Proof.
Similar to Theorem2, by introducing Lagrange multipliers and considering the constraint ∑r=1Rwr=1, we have Theorem 3.Algorithm 1: Descriptions of the Co-MVFCM method.
INPUT: Multiview datasetDatar=X1,⋯,XN. Number of clusters C2≤C<N, stop threshold ε, fuzzy index m and p, number of iteration f, maximum number of iteration L, regularization parameter λOUTPUT: Center of each clustervi,r, global fuzzy membership degree matrix U, view weight wrStep 1. Randomize vi, μij, and wrStep 2. Update vi,r based on Equation (12)Step 3. Update μij,r based on Equation (13)Step 4. Update wr based on Equation (18)Step 5. If Jl+1−Jl<ε or f>L, stop loop; otherwise, go to Step2Step 6. Compute U~ based on Equation (11)Remark 4.
In this section, a novel multiview clustering method called Co-MVFCM is proposed. The proposed Co-MVFCM method can find the most important view adaptively, and it also can obtain the best space division by using Equation (11). However, we will find that the proposed Co-MVFCM method has three predefined parameters. These predefined parameters should be defined by using grid optimization which will lose many time costs. In the near future, we will consider how to reduce the number of these predefined parameters.


## 3.1. Approximation Criterion of Space Division from Different Views Based on the Havrda-Charvat Entropy

In this study, the Havrda-Charvat entropy ofxj,r is defined as
(6)Ψmxj,r=11−21−m∑i=1Cμij,rm−1.It is obvious that if the fuzzy membership degreeUr=μij,rm is considered as a probability matrix, when the constraint ∑i=1Cμij,r=1 holds, Ψmxj,r is equal to 0. It is very intuitive to show that the uncertainty of xj,r belonging to each division in the sample set Xr of this view reaches the minimum value. That is to say, when the objective function reaches its minimum value, the Havrda-Charvat entropy of μij,r also reaches its minimum value.Although Equation (6) can ensure that the uncertainty of division can be minimized, it is limited to a single view. In order to expand it into a field of multiple views, in this study, we expand Equation (6) into the following expression form by referring to the relevant strategies used in [20]:
(7)Φ=11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1R−1∑i=1C∑j=1N∑r′=1,r′≠rRμij,r′m−1−μij,rm−1.So we have(8)Φ=1−β11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1K−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1.We observe from Equations (7) and (8) that η can be used to effectively regulate the weight relationship between the current view and the membership degree division of other views (0<β<1 and β=R−1/R). So we can get the weighted average of membership degree and finally make the membership degree division of each view as consistent as possible, so as to obtain the spatial division result with a more global view.


## 3.2. Multiview Adaptive Weighting Based on Fuzzy Index

In this study, we develop an automatic view weighting strategy using fuzzy index to recognize the best view. Supposewr represents the weight of view r under the condition that ∑r=1Rwr=1 and wr≥0, then wr can be considered as the probability distribution which is defined as
(9)fwr=∑r=1Rwrp.Fuzzy index technology is introduced through the above methods to make the objective function achieve the optimal entropy as much as possible, which is also the classical fuzzy c-means clustering principle [4].


## 3.3. Co-MVFCM

According to the above definitions, we propose our new multiview clustering method. The objective function of Co-MVFCM is(10)JCo−MVFCM=∑r=1Rwrp∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr,Φr=1−β11−21−m∑i=1C∑j=1Nμij,rm−1+β11−21−m1R−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1s.t.μij,r∈0,1,∑i=1Cμij,r=1 and ∑r=1Rwr=1, 1≤r≤R, 1≤j≤N.Obviously, the objective function contains two main parts. The first one isℓU,V,X=∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr which is derived from Havrda-Charvat and used for collaborative clustering. The essence of the first part is to find out as many similar parts among different perspectives as possible through multiview clustering technology and finally make the spatial division results of different views tend to be the same. The second part is ℓ~U,V,X,W=∑r=1RwrpℓU,V,X which is derived from fuzzy index. This part can be used to adaptively calculate the weight values of each view, and finally, when the algorithm reaches the optimal level, the optimal view partitioning results can be obtained according to the weight matrix of the views. The parameter β can be set to β=R−1/R. The parameter λ can be determined by using grid optimization [13, 21].To obtain the final result of space division with global characteristics, the integration strategy of global space division mentioned in [20] is abandoned in this study. We define a new integration strategy to obtain the final space division as
(11)U~=∑r=1RwrUrR.


### 3.3.1. Optimization

The proposed multiview can be optimized by introducing Lagrange multipliers. In this section, we give three theorems to obtain updating rules in terms of fuzzy membership degree, view weights, and cluster centers.Theorem 1.
WhenUr and Wr are fixed, the cluster center can be solved by
(12)vi,r=∑j=1Nμij,rmxj,r∑j=1Nμij,rm.Proof.
By setting∂J/∂Vr=0, we have vi,r=∑j=1Nμij,rmxj,r/∑j=1Nμij,rm. Therefore, Theorem 1 is achieved.Theorem 2.
When the cluster centerVr and view weight matrix Wr are fixed, the fuzzy membership degree matrix can be solved by
(13)μij,r=1∑h=1C1−β/1−21−mλ+xj,r−vi,r2/1−β/1−21−mλ+xj,r−vh,r21/m−1.Proof.
By introducing Lagrange multipliersαj,r and considering the constraint ∑i=1Cμij,r=1, we have the following objective function:
(14)JCo−MVFCMμij,r,βj,r=−∑r=1R∑j=1Nοj,r∑i=1Cμij,r−1+∑r=1Rwrp∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr,where Φr=β1/1−21−m1/R−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1+1−β1/1−21−m∑i=1C∑j=1Nμij,rm−1.
By setting the partial derivative ofJCo−MVFCMμij,r,οj,r w.r.t. μij,r,οj,r to 0, i.e., ∂JCo−MVFCM/∂μij,r=0, we have
(15)μij,r=οj,rmwrp1−β/1−21−mλ+xj,r−vi,r21/m−1.
Similarly, with∂JCo−MVFCM/∂οj,r=0, we have
(16)∑i=1Cμij,r=1.
By combining Equations (15) and (16) to remove οj,r, we have
(17)μij,r=1∑h=1Cxj,r−vi,r2+1−β/1−21−mλ/xj,r−vh,r2+1−β/1−21−mλ1/m−1.
Therefore, the proof of Theorem2 is achieved.Theorem 3.
When the center matrixVr and the fuzzy membership degree matrix Ur are fixed, the weight matrix can be solved by
(18)wr=1∑r′=1R∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr/∑i=1C∑j=1Nμij,r′mxj,r′−vi,r′2+λΦr′1/p−1.Proof.
Similar to Theorem2, by introducing Lagrange multipliers and considering the constraint ∑r=1Rwr=1, we have Theorem 3.Algorithm 1: Descriptions of the Co-MVFCM method.
INPUT: Multiview datasetDatar=X1,⋯,XN. Number of clusters C2≤C<N, stop threshold ε, fuzzy index m and p, number of iteration f, maximum number of iteration L, regularization parameter λOUTPUT: Center of each clustervi,r, global fuzzy membership degree matrix U, view weight wrStep 1. Randomize vi, μij, and wrStep 2. Update vi,r based on Equation (12)Step 3. Update μij,r based on Equation (13)Step 4. Update wr based on Equation (18)Step 5. If Jl+1−Jl<ε or f>L, stop loop; otherwise, go to Step2Step 6. Compute U~ based on Equation (11)Remark 4.
In this section, a novel multiview clustering method called Co-MVFCM is proposed. The proposed Co-MVFCM method can find the most important view adaptively, and it also can obtain the best space division by using Equation (11). However, we will find that the proposed Co-MVFCM method has three predefined parameters. These predefined parameters should be defined by using grid optimization which will lose many time costs. In the near future, we will consider how to reduce the number of these predefined parameters.


## 3.3.1. Optimization

The proposed multiview can be optimized by introducing Lagrange multipliers. In this section, we give three theorems to obtain updating rules in terms of fuzzy membership degree, view weights, and cluster centers.Theorem 1.
WhenUr and Wr are fixed, the cluster center can be solved by
(12)vi,r=∑j=1Nμij,rmxj,r∑j=1Nμij,rm.Proof.
By setting∂J/∂Vr=0, we have vi,r=∑j=1Nμij,rmxj,r/∑j=1Nμij,rm. Therefore, Theorem 1 is achieved.Theorem 2.
When the cluster centerVr and view weight matrix Wr are fixed, the fuzzy membership degree matrix can be solved by
(13)μij,r=1∑h=1C1−β/1−21−mλ+xj,r−vi,r2/1−β/1−21−mλ+xj,r−vh,r21/m−1.Proof.
By introducing Lagrange multipliersαj,r and considering the constraint ∑i=1Cμij,r=1, we have the following objective function:
(14)JCo−MVFCMμij,r,βj,r=−∑r=1R∑j=1Nοj,r∑i=1Cμij,r−1+∑r=1Rwrp∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr,where Φr=β1/1−21−m1/R−1∑r′=1,r′≠rR∑i=1C∑j=1Nμij,r′m−1+1−β1/1−21−m∑i=1C∑j=1Nμij,rm−1.
By setting the partial derivative ofJCo−MVFCMμij,r,οj,r w.r.t. μij,r,οj,r to 0, i.e., ∂JCo−MVFCM/∂μij,r=0, we have
(15)μij,r=οj,rmwrp1−β/1−21−mλ+xj,r−vi,r21/m−1.
Similarly, with∂JCo−MVFCM/∂οj,r=0, we have
(16)∑i=1Cμij,r=1.
By combining Equations (15) and (16) to remove οj,r, we have
(17)μij,r=1∑h=1Cxj,r−vi,r2+1−β/1−21−mλ/xj,r−vh,r2+1−β/1−21−mλ1/m−1.
Therefore, the proof of Theorem2 is achieved.Theorem 3.
When the center matrixVr and the fuzzy membership degree matrix Ur are fixed, the weight matrix can be solved by
(18)wr=1∑r′=1R∑i=1C∑j=1Nμij,rmxj,r−vi,r2+λΦr/∑i=1C∑j=1Nμij,r′mxj,r′−vi,r′2+λΦr′1/p−1.Proof.
Similar to Theorem2, by introducing Lagrange multipliers and considering the constraint ∑r=1Rwr=1, we have Theorem 3.Algorithm 1: Descriptions of the Co-MVFCM method.
INPUT: Multiview datasetDatar=X1,⋯,XN. Number of clusters C2≤C<N, stop threshold ε, fuzzy index m and p, number of iteration f, maximum number of iteration L, regularization parameter λOUTPUT: Center of each clustervi,r, global fuzzy membership degree matrix U, view weight wrStep 1. Randomize vi, μij, and wrStep 2. Update vi,r based on Equation (12)Step 3. Update μij,r based on Equation (13)Step 4. Update wr based on Equation (18)Step 5. If Jl+1−Jl<ε or f>L, stop loop; otherwise, go to Step2Step 6. Compute U~ based on Equation (11)Remark 4.
In this section, a novel multiview clustering method called Co-MVFCM is proposed. The proposed Co-MVFCM method can find the most important view adaptively, and it also can obtain the best space division by using Equation (11). However, we will find that the proposed Co-MVFCM method has three predefined parameters. These predefined parameters should be defined by using grid optimization which will lose many time costs. In the near future, we will consider how to reduce the number of these predefined parameters.


## 4. Experimental Studies

### 4.1. Settings

In this study, we introduce several UCI datasets to evaluate the proposed multiview clustering method. For fair comparison, Co-FKM [20], LSSMTC [22], CombKM [22], and Coclustering [23] are introduced for benchmarking testing.We introduce two commonly used criteria, i.e., NMI and RI to evaluate all clustering methods. They are defined as follows.(1)
Normalized Mutual Information (NMI) [24, 25]
(19)NMI=∑i=1C∑j=1CNi,jlogN⋅Ni,j/Ni⋅Nj∑i=1CNilogNi/N⋅∑j=1CNjlogNj/N,where Ni represents the number of samples in the ith cluster, Ni,j represent the matching degree of the ith cluster and the jth cluster, and N represents the size of the dataset.(2)
Rand Index (RI) [24, 25]
(20)RI=f00+f11NN−1/2,where f11 represents the number of pairing points that have the same class label and belong to the same class and f00 represents the number of matching points with different class labels and belonging to different classes of data points.The value range of the above two indexes is [0 1]. The closer the value of these two indicators is to 1, the better the performance is. Experimental environment: the experimental hardware platform was Intel Core i7 CPU, with a memory of 16 GB. The programming environment is MATLAB 2010.


### 4.2. Experimental Results

In this section, some real-world datasets from the famous UCI database will be used to test our algorithm: (1) Iris dataset, (2) Multiple Features (MF) dataset, (3) Image Segmentation (IS) dataset, and (4) Water Treatment Plant (WTP) dataset. The performance of the Co-MVFCM algorithm proposed in this study is verified and analyzed by using the above datasets when processing real multiview clustering tasks. In order to have a more intuitive impression of the perspectives contained in the three datasets, this paper will present the composition of the four datasets, as shown in Table1. At the same time, the experimental results of algorithm comparison for these four real datasets are shown in Table 2.Table 1
The description of some UCI datasets.
DatasetsDescription of these datasetsThe number of the dimensionsThe number of the clustersThe number of the samplesIrisClasses ofIris plants43150Multiple Features (MF)Handwritten digits649102000Image Segmentation (IS)Outdoor images1972310Water Treatment Plant (WTP)Urban wastewater treatment dataset3813527Table 2
Comparison of performance indices NMI and RI of several algorithms on the datasets.
DatasetAlgorithmsNMI-meanNMI-stdRI-meanRI-stdIrisLSSMTC0.72600.04700.88790.0219CombKM0.74120.05370.86360.0511Coclustering0.75821.17e−0160.87970Co-FKM0.83661.48e−0160.93410Co-MVFCM0.86660.00050.94880.0045MFLSSMTC————CombKM0.68830.03370.89910.0119Coclustering0.70240.02870.91040.0126Co-FKM0.78680.07150.94170.0226Co-MVFCM0.78920.02170.95330.0089ISLSSMTC————CombKM0.48150.02830.79810.0176Coclustering0.51820.01050.81890.0208Co-FKM0.54220.03530.82930.0195Co-MVFCM0.59020.02170.84390.0174WTPLSSMTC————CombKM0.17610.01210.70420.0037Coclustering0.20290.01030.70510.0061Co-FKM0.19860.01400.70560.0044Co-MVFCM0.22040.01020.72090.0035For the Iris dataset, we will observe that the proposed Co-MVFCM has the best clustering performance among all the adopted comparison algorithms. The experimental result of Iris shows that the proposed two multiview collaborative clustering strategies have significant advantages in multiview clustering task. For the other three datasets, since the LSSMTC algorithm needs to ensure that the dimensions of each clustering task are consistent, it cannot be used in the face of the samples with different perspectives such as MF, IS, and WTP. By observing the rest of each other algorithm’s clustering results of MF datasets, it can be found that based on the multiple points of view of Co-FKM, the algorithm of this paper has a larger cluster advantage, but because of MF data, no angle exists obvious separability which exist the importance degree of the equilibrium between different points of view; this makes the clustering results from the NMI and RI of the proposed algorithm in the paper with Co-FKM algorithm similar to the average of the two major indicators, and from the variance analysis, the method is still more stable so it still reflects that the method still has certain advantages. For IS dataset, the effect of the proposed method on this sample is relatively obvious, and its clustering index is significantly better than that of the other algorithms, which further confirms the effectiveness of the Co-MVFCM. Finally, through the analysis of the experimental results of the WTP dataset, the same conclusion can be obtained with the above two datasets. In conclusion, through the experiments on real dataset multiple points of view and analysis, we can get a clear conclusion of the clustering algorithm in dealing with multiple points of view which have many view feature clustering tasks generally superior to the clustering algorithm, multiple points of view, and has a view of selective Co-MVFCM algorithm, and clustering algorithm is much better than the previous view.


## 4.1. Settings

In this study, we introduce several UCI datasets to evaluate the proposed multiview clustering method. For fair comparison, Co-FKM [20], LSSMTC [22], CombKM [22], and Coclustering [23] are introduced for benchmarking testing.We introduce two commonly used criteria, i.e., NMI and RI to evaluate all clustering methods. They are defined as follows.(1)
Normalized Mutual Information (NMI) [24, 25]
(19)NMI=∑i=1C∑j=1CNi,jlogN⋅Ni,j/Ni⋅Nj∑i=1CNilogNi/N⋅∑j=1CNjlogNj/N,where Ni represents the number of samples in the ith cluster, Ni,j represent the matching degree of the ith cluster and the jth cluster, and N represents the size of the dataset.(2)
Rand Index (RI) [24, 25]
(20)RI=f00+f11NN−1/2,where f11 represents the number of pairing points that have the same class label and belong to the same class and f00 represents the number of matching points with different class labels and belonging to different classes of data points.The value range of the above two indexes is [0 1]. The closer the value of these two indicators is to 1, the better the performance is. Experimental environment: the experimental hardware platform was Intel Core i7 CPU, with a memory of 16 GB. The programming environment is MATLAB 2010.


## 4.2. Experimental Results

In this section, some real-world datasets from the famous UCI database will be used to test our algorithm: (1) Iris dataset, (2) Multiple Features (MF) dataset, (3) Image Segmentation (IS) dataset, and (4) Water Treatment Plant (WTP) dataset. The performance of the Co-MVFCM algorithm proposed in this study is verified and analyzed by using the above datasets when processing real multiview clustering tasks. In order to have a more intuitive impression of the perspectives contained in the three datasets, this paper will present the composition of the four datasets, as shown in Table1. At the same time, the experimental results of algorithm comparison for these four real datasets are shown in Table 2.Table 1
The description of some UCI datasets.
DatasetsDescription of these datasetsThe number of the dimensionsThe number of the clustersThe number of the samplesIrisClasses ofIris plants43150Multiple Features (MF)Handwritten digits649102000Image Segmentation (IS)Outdoor images1972310Water Treatment Plant (WTP)Urban wastewater treatment dataset3813527Table 2
Comparison of performance indices NMI and RI of several algorithms on the datasets.
DatasetAlgorithmsNMI-meanNMI-stdRI-meanRI-stdIrisLSSMTC0.72600.04700.88790.0219CombKM0.74120.05370.86360.0511Coclustering0.75821.17e−0160.87970Co-FKM0.83661.48e−0160.93410Co-MVFCM0.86660.00050.94880.0045MFLSSMTC————CombKM0.68830.03370.89910.0119Coclustering0.70240.02870.91040.0126Co-FKM0.78680.07150.94170.0226Co-MVFCM0.78920.02170.95330.0089ISLSSMTC————CombKM0.48150.02830.79810.0176Coclustering0.51820.01050.81890.0208Co-FKM0.54220.03530.82930.0195Co-MVFCM0.59020.02170.84390.0174WTPLSSMTC————CombKM0.17610.01210.70420.0037Coclustering0.20290.01030.70510.0061Co-FKM0.19860.01400.70560.0044Co-MVFCM0.22040.01020.72090.0035For the Iris dataset, we will observe that the proposed Co-MVFCM has the best clustering performance among all the adopted comparison algorithms. The experimental result of Iris shows that the proposed two multiview collaborative clustering strategies have significant advantages in multiview clustering task. For the other three datasets, since the LSSMTC algorithm needs to ensure that the dimensions of each clustering task are consistent, it cannot be used in the face of the samples with different perspectives such as MF, IS, and WTP. By observing the rest of each other algorithm’s clustering results of MF datasets, it can be found that based on the multiple points of view of Co-FKM, the algorithm of this paper has a larger cluster advantage, but because of MF data, no angle exists obvious separability which exist the importance degree of the equilibrium between different points of view; this makes the clustering results from the NMI and RI of the proposed algorithm in the paper with Co-FKM algorithm similar to the average of the two major indicators, and from the variance analysis, the method is still more stable so it still reflects that the method still has certain advantages. For IS dataset, the effect of the proposed method on this sample is relatively obvious, and its clustering index is significantly better than that of the other algorithms, which further confirms the effectiveness of the Co-MVFCM. Finally, through the analysis of the experimental results of the WTP dataset, the same conclusion can be obtained with the above two datasets. In conclusion, through the experiments on real dataset multiple points of view and analysis, we can get a clear conclusion of the clustering algorithm in dealing with multiple points of view which have many view feature clustering tasks generally superior to the clustering algorithm, multiple points of view, and has a view of selective Co-MVFCM algorithm, and clustering algorithm is much better than the previous view.


## 5. Conclusion

Based on cluster technology, multiple points of view are introduced on the basis of the classical FCM algorithm using the Havrda-Charvat entropy structure of different view space approaching. The proposed Co-MVFCM method can better find out the similarities between view compositions, but also from the view of entropy approximation of a different view space, reasonable physical explanation, and thus get more guiding significance to the overall space partition. In addition, in this paper, another contribution is to obtain the importance degree of each view. Through the understanding of the fuzzy theory, multiple points of view are proposed based on the fuzzy index of the adaptive weighted strategy and succeeded in introducing the strategy to the latest fuzzy clustering technology, multiple points of view on the new objective function to achieve the optimal solution. Next, we can evaluate the degree of importance of each view according to the relationship between the weights of each view. The obtained degree of importance of each view provides a new method of integration of the global weighted view space integration means. Experimental results on four real UCI datasets show that the Co-MVFCM has better sample adaptability and superior algorithm performance compared with previous algorithms and related algorithms. However, since the algorithm in this paper is based on the framework of the classical fuzzy c-means (FCM) algorithm, the effectiveness of the algorithm may be tested to a certain extent when dealing with higher-dimensional data, which also points out the direction for our future research on the multiview clustering method under high-dimensional data scene.



---

*Source: 9963133-2021-05-03.xml*