Improved cluster validity index for fuzzy clustering

Electronics LettersVolume 57, Issue 21 p. 792-794 Information and communicationsOpen Access Improved cluster validity index for fuzzy clustering Soon Hak Kwon, Soon Hak Kwon Department of Electrical Engineering, Yeungnam University, Gyeongsan, Republic of KoreaSearch for more papers by this authorJihong Kim, Corresponding Author Jihong Kim [email protected] Department of Electrical Engineering, Yeungnam University, Gyeongsan, Republic of Korea Email: [email protected]Search for more papers by this authorSeo Ho Son, Seo Ho Son Top Engineering Co., Ltd., Seongnam, Republic of KoreaSearch for more papers by this author Soon Hak Kwon, Soon Hak Kwon Department of Electrical Engineering, Yeungnam University, Gyeongsan, Republic of KoreaSearch for more papers by this authorJihong Kim, Corresponding Author Jihong Kim [email protected] Department of Electrical Engineering, Yeungnam University, Gyeongsan, Republic of Korea Email: [email protected]Search for more papers by this authorSeo Ho Son, Seo Ho Son Top Engineering Co., Ltd., Seongnam, Republic of KoreaSearch for more papers by this author First published: 04 September 2021 https://doi.org/10.1049/ell2.12249AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract An improved cluster validity index for fuzzy clustering that is able to overcome three intrinsic drawbacks in conventional cluster validity indexes is proposed. The effectiveness of the proposed index is demonstrated by the analysis of its limiting behaviour and comparison with the performance of the Xie-Beni index, Kwon index, and Tang index on twelve data sets. Introduction One of the most important issues in fuzzy clustering is the validation of the clustering results. Therefore, many research results on the validation have been presented and widely used in various areas. However, the conventional fuzzy cluster validity indexes including Xie-Beni index [1], Kwon index [2], and Tang index [3] have well-known drawbacks, respectively, that is, (i) monotonically decreasing tendency occurring when the number of cluster c gets large and close to the number of data points n, (ii) numerical instability due to its limiting behaviour when fuzzy weighting exponent m approaches infinity, and (iii) under-clustering tendency that the estimated number of clusters is smaller than the true number of clusters. In this letter, a robust index for fuzzy cluster validation that is able to overcome the above three intrinsic drawbacks is proposed. The limiting behaviour of the proposed index is analyzed, and the performance comparison with the Xie-Beni index, Kwon index, and Tang index on twelve numerical examples is also provided to prove the effectiveness. Conventional fuzzy cluster validity indexes In this sequel to our previous paper [2], we use the following notation: Let p × n matrix X = { x1, x2,…, xn} ⊂ Rp denote a given data set which are n points in the p-dimensional space. The data matrix X has c fuzzy clusters C = {C1, C2,…, Cc} of which cluster centres are given by matrix V = [v1,v1,…, vc] where c ∈ (1, n) is the number of clusters and the membership matrix U = [μij]c × n where μij is the membership value of data point xj. First, we consider the Xie-Beni index VXB: V X B U , V ; X = ∑ j = 1 n ∑ i = 1 c μ i j 2 x j − v i 2 n min i ≠ k v i − v k 2 , (1)and the extended Xie-Beni index [1]: V X B U , V ; X = ∑ j = 1 n ∑ i = 1 c μ i j m x j − v i 2 n min i ≠ k v i − v k 2 . (2) VXB decreases monotonically when the number of clusters c is close to n. To overcome the drawback, Kwon proposed the Kwon index VK [2]: V K U , V ; X = ∑ j = 1 n ∑ i = 1 c μ i j 2 x j − v i 2 + 1 c ∑ i = 1 c v i − v ¯ 2 min i ≠ k v i − v k 2 (3)where v ¯ = 1 n ∑ j = 1 n x j . As Tang et al. pointed out, VK has a drawback of numerical instability as fuzzy weighting exponent m approaches infinity. To overcome the drawback, Tang et al. proposed the Tang index VT [3]: V T U , V ; X = ∑ i = 1 c ∑ j = 1 n μ i j 2 x j − v i 2 + 1 c c − 1 ∑ i = 1 c ∑ k = 1 , k ≠ i c v i − v k 2 m i n i ≠ k v i − v k 2 + 1 c . (4) The limiting behaviour of VXB, VK, and VT when c → n and m → ∞ can be found in [1-3]. Even though Tang's trial to strengthen the numerical stability by introducing a punishing function, that is, the second term in the denominator of (4), it cannot provide the numerical stability as m → ∞. In addition to the above problem, VK and VT show the under-clustering tendency as stated in the introduction. Improved cluster validity index for fuzzy clustering In clustering, it is usual to attempt to maximize intra-class similarity and inter-class differences. In order to overcome the three intrinsic problems of the conventional fuzzy cluster validity indexes stated in the introduction, we propose an improved fuzzy cluster validity index VK2 as follows: V K 2 U , V ; X = w 1 w 2 ∑ i = 1 c ∑ j = 1 n μ i j 2 m 2 x j − v i 2 + ∑ i = 1 c v i − v ¯ 2 m a x i v i − v ¯ 2 + w 3 min i ≠ k v i − v k 2 + 1 c + 1 c m − 1 (5)where w 1 = n − c + 1 n , w 2 = ( c c − 1 ) 2 , and w 3 = n c ( n − c + 1 ) 2 . The optimal cluster number c* can be obtained by searching c minimizing the value of VK2. The first term of the numerator in (5) denotes the inter-class differences of data set X with respect to c fuzzy clusters C = {C1, C2,…, Cc} . The second term and the last term of the numerator indicate a punishing function applied to eliminate the decreasing tendency as c → n and a dummy term for the stability of the proposed index when c → n and m → ∞, respectively. The second term of the denominator in (5) is ad hoc punishing function used to strengthen the numerical stability as m → ∞. The last term of the denominator is ad hoc punishing function used to strengthen the numerical stability as c → n. To investigate the limiting behaviour of the proposed index, we take two limits of the validation index when c → n and m → ∞. Since lim c → n v i = x i and lim c → n ∥ x j − v i ∥ 2 = 0 , we obtain lim c → n V K 2 U , V ; X = 1 n C X m a x i x i − v ¯ 2 + n 2 min i ≠ k x i − x k 2 + 1 n + 1 n m − 1 (6)where CX is the total scatter matrix defined by C X = ∑ j = 1 n ∥ x j − v ¯ ∥ 2 . On the other hand, since lim m → ∞ μ i j = 1 c , lim m → ∞ v i = v ¯ , lim m → ∞ ∥ v i − v ¯ ∥ 2 = 0 , and lim m → ∞ ∥ v i − v k ∥ 2 = 0 , we get lim m → ∞ V K 2 U , V ; X = c w 1 w 3 = c 2 n − c + 1 (7) Finally, when c → n and m → ∞, we obtain lim c → n m → ∞ V K 2 U , V ; X = C X m a x i x i − v ¯ 2 + n 2 (8) From (6), (7) and (8), we can see that the proposed validation index can keep the ability to validate (U, V) pairs as c → n or m → ∞, and as c → n and m → ∞ simultaneously. That is to say, the proposed validation index can provide a similar or better response over a wide range of choices for the number of clusters and fuzzy weighting exponent compared to VXB, VK, and VT. Computer simulation results and remarks In this section, we consider twelve data sets to show the effectiveness of the proposed fuzzy cluster validity index. All simulations were performed in MATLAB R2019b on a computer with the following configuration: Intel(R) Core (TM) i5-4590 CPU @3.30GHz and 8GB of memory with Microsoft Windows 10 system. Two types of twelve data sets, i.e. artificial data sets (X30 [4], Dim2, b2-sub-2, b2-sub-3, b2-sub-4, b2-sub-5, and b2-sub-9 [5]) and data sets from the UCI repository (Iris, Seeds, Yeast, and Wine) were tested for the performance evaluation of the various indexes [6]. The data distributions of the artificial data sets with the number of features of two are depicted in Figures 1-8. The specification of all data sets is detailed in Table 1. Fig. 1Open in figure viewerPowerPoint Tested data set: Dim2 Fig. 2Open in figure viewerPowerPoint Tested data set: X30 Fig. 3Open in figure viewerPowerPoint Tested data set: b2-sub-2 Fig. 4Open in figure viewerPowerPoint Tested data set: b2-sub-3 Fig. 5Open in figure viewerPowerPoint Tested data set: b2-sub-4 Fig. 6Open in figure viewerPowerPoint Tested data set: b2-sub-5 Fig. 7Open in figure viewerPowerPoint Tested data set: b2-sub-9 Fig. 8Open in figure viewerPowerPoint Tested data set: b2-sub-10 Table 1. Details of considered data sets Data set Number of instances Number of features True cluster numbers Iris 150 4 2 or 3 Seeds 210 8 3 Yeast 1484 8 10 wine 178 13 3 Dim2 1351 2 9 X30 30 2 3 b2-sub-2 2000 2 2 b2-sub-3 3000 2 3 b2-sub-4 4000 2 4 b2-sub-5 5000 2 5 b2-sub-9 9000 2 9 b2-sub-10 10000 2 10 In the simulations, the maximum number of clusters was set to 2 ≤ r o u n d ( n ) ≤ 11 for all data sets. The interference of randomness in fuzzy c-means (FCM) was minimized by executing 100 rounds of FCM on each data set [7]. The weighting exponent m can influence the performance of FCM. In practical applications, it is generally suggested that the optimal range of m is within [1.5, 2.5]. Tables 2–4 show the optimal number of clusters obtained from each index when m changes. Table 2. Clustering results on considered data sets in terms of cluster numbers decided by various indexes (m = 2) Data set c* VXB VK VT VK2 Iris 2 or 3 2 2 2 2 Seeds 3 2 2 2 3 Yeast 10 2 2 2 10 Wine 3 3 3 3 3 Dim2 9 9 9 9 9 X30 3 3 2 2 3 b2-sub-2 2 2 2 2 2 b2-sub-3 3 3 3 3 3 b2-sub-4 4 4 4 4 4 b2-sub-5 5 5 5 5 5 b2-sub-9 9 9 9 9 9 b2-sub-10 10 10 10 10 10 Table 3. Clustering results on considered data sets in terms of cluster numbers decided by various indexes (m = 6) Data set c* VXB VK VT VK2 Iris 2 or 3 2 2 2 2 Seeds 3 2 2 2 3 Yeast 10 9 9 2 7 Wine 3 2 2 2 2 Dim2 9 9 9 9 9 X30 3 2 2 2 3 b2-sub-2 2 2 2 2 2 b2-sub-3 3 2 2 2 3 b2-sub-4 4 2 2 2 4 b2-sub-5 5 2 2 2 5 b2-sub-9 9 2 2 2 9 b2-sub-10 10 2 2 2 10 Table 4. Clustering results on considered data sets in terms of cluster numbers decided by various indexes (m = 10) Data set c* VXB VK VT VK2 Iris 2 or 3 2 2 2 2 Seeds 3 2 2 2 3 Yeast 10 4 4 2 5 Wine 3 11 11 11 11 Dim2 9 3 3 3 5 X30 3 2 2 2 3 b2-sub-2 2 2 2 2 2 b2-sub-3 3 2 2 2 3 b2-sub-4 4 2 2 2 4 b2-sub-5 5 2 2 2 5 b2-sub-9 9 2 2 2 6 b2-sub-10 10 2 2 2 7 As expected, while VK and VT exhibit strong under-clustering tendency when m becomes large, the results from the proposed index VK2 show a weak tendency. From these results, it can be concluded that the performance of the proposed index VK2 obviously outperforms those of the conventional VXB, VK, and VT. Conclusion In this letter, we proposed an improved cluster validity index for fuzzy clustering and provided its limiting behaviour. We also carried out the computer simulation on twelve numerical examples to show the effectiveness of the index. From the simulation results, we can conclude that the proposed fuzzy cluster validity index VK2 outperforms the conventional VXB, VK, and VT. Further research and application to diverse areas will be expected. Acknowledgments This research was supported by Yeungnam University (219A580083). References 1Xie, X.L., Beni, G.: A validity measure for fuzzy clustering. IEEE Trans. Pattern Anal. Mach. Intell. 13(8), 841– 847 (1991) https://doi.org/10.1109/34.85677 2Kwon, S. H.: Cluster validity index for fuzzy clustering. Electron. Lett. 34(22), 2176– 2177 (1998) https://doi.org/10.1049/el:19981523 3Tang, Y., et al.: Improved validation index for fuzzy clustering. In: Proceedings of the 2005, American Control Conference, Portland, OR, USA, 8–10 June 2005. https://doi.org/10.1109/ACC.2005.1470111 4Bezdek, J.C., Pal, N.R.: Some new indexes of cluster validity. IEEE Trans. Syst., Man Cybern., Part B (Cybern.) 28, (3), 301– 315 (1998). https://doi.org/10.1109/3477.678624 5Franti, P., Sieranoja, S.: K-means properties on six clustering benchmark datasets. Appl. Intell. 48, 4743– 4759 (2018). https://doi.org/10.1007/s10489-018-1238-7 6UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science 7Mittal, H., Saraswat, M.: A new fuzzy cluster validity index for hyper-ellipsoid or hyper-spherical shape close clusters with distant centroids. IEEE Trans. Fuzzy Syst. 1 (2020). https://doi.org/10.1109/TFUZZ.2020.3016339 Volume57, Issue21October 2021Pages 792-794 FiguresReferencesRelatedInformation