Fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm
Linear Discriminant Analysis (LDA) is a classical statistical approach for supervised feature extraction and dimensionality reduction, hard c-means (HCM) is a classical unsupervised learning algorithm for clustering. Based on the analysis of the relationship between LDA and HCM, Linear Discriminant Analysis-guided adaptive subspace hard c-means clustering algorithm (LDAHCM) had been proposed. LDAHCM combines LDA and HCM into a coherent framework and can adaptively reduce the dimension of data while performing data clustering simultaneously. Seeing that LDAHCM is still a hard clustering algorithm, we consider the fuzzy extension version of LDAHCM in this paper. To this end, firstly, we propose a new optimization criterion of Fuzzy Linear Discriminant Analysis (FLDA) by extending the value of membership function in classical LDA from binary 0 or 1 into closed interval [0, 1]. In the meantime, we present an efficient algorithm for the proposed FLDA. Secondly, we show the close relationship between FLDA and Maximum Entropy Fuzzy Clustering Algorithm (MEFCA): they both are maximizing fuzzy between-class scatter and minimizing within-class scatter simultaneously. Finally, based on the above analysis, combining FLDA and MEFCA into a joint framework, we propose fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm (FLDAMEFCA). LDAMEFCA is a natural and effective fuzzy extension of LDAHCM. Due to the introduction of soft decision strategy, FLDAMEFCA can yield fuzzy partition of data set and is more flexible than LDAHCM. We also give the convergence proof of FLDAMEFCA. Extensive experiments on a collection of benchmark data sets are presented to show the effectiveness of the proposed algorithm.
fuzzy linear discriminant analysis maximum entropy fuzzy clustering algorithm fuzzy scatter matrix optimal transformation matrix