# An Accurate Method of Determining Attribute Weights in Distance-Based Classification Algorithms

Authors: Fengtao Liu; Jialei Wang
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Weight determination aims to determine the importance of different attributes; determining accurate weights can significantly improve the accuracy of classification and clustering. This paper proposes an accurate method for attribute weight determination. The method uses the distance from the sample point of each class to the class center point. It can minimize the weights and determines the attribute weights of the constraints through the objective function. In this paper, the attribute weights obtained by the exact solution are applied to the K-means clustering algorithm; three classic machine learning data sets, the iris data set, the wine data set, and the wheat seed data set, are clustered. Using the normalized mutual information as the evaluation index, a confusion matrix was established. Finally, the clustering results are visualized and compared with other methods to verify the effectiveness of the proposed method. The results show that this method improves the normalized mutual information by 0.11 and 0.08, respectively, compared with the unweighted and entropy weighted methods for iris clustering results. Furthermore, the performance on the wine data set is improved by 0.1, and the performance on the wheat seed data set is improved by 0.15 and 0.05.


---

## Body

## 1. Introduction

Weights reflect the importance of different attributes, and the influence of different attribute weights on algorithm results is sometimes very different. It is necessary to determine accurate attribute weights. Let us take K-means as an example. K-means clustering is a typical distance-based clustering algorithm. K-means is widely used due to its fast-running speed, simplicity, and ease of understanding. However, traditional K-means does not consider the importance of features, resulting in poor clustering effects with traditional K-means in some problems. The distance class algorithm uses the distance between sample attributes to classify and cluster [1, 2]. Generally, the sample cluster is divided by clustering birds of a feather [3, 4] to achieve the effect of high similarity within the cluster and low similarity outside the cluster [5]. The distance between sample attributes is a “distance measure” [6, 7]. The similarity measure defined by us means that the larger the distance, the smaller the similarity [8, 9]. Differences between different attributes may not be obvious or even wrong in some distance performance, which can be achieved through “distance metric learning.” In other words, assigning different weights to sample attributes improves learning effects [10].At present, the problem of weight determination can be divided into two methods: subjective weight determination and objective weight determination. Domain experts compare the importance degree of each attribute with fuzzy language to determine the weight. The methods of subjective weight determination by experts include the analytic hierarchy process (AHP), sequence diagram method, simple weighting, etc. The analytic hierarchy process is a widely used method at present. Pourghasemi et al. used fuzzy logic and an analytic hierarchy process (AHP) model to make a landslide sensitivity map of Iran’s landslide-prone area (Haraz) for land planning and disaster reduction [11]. Lin and Kou [12], based on the multiplication AHP model, proposed a heuristic method, and priority vectors were derived from the PCM in the whole hierarchy.Although the subjective weight determination method has achieved good results in some conditions, it is limited by the shortcomings of artificial judgment, inability to find experts, and so on. Therefore, the objective weight determination method is used in many cases. The methods of objective weight mainly include the entropy weight method, principal component analysis method, and factor analysis. Meimei et al. proposed two methods to determine the optimal weight of attributes based on entropy and measure [13]. Chen combined the entropy weight method with Topsis to determine the weight of Topsis attributes and analyzed the influence of electronic warfare on Topsis [14]. Amaya et al. proposed a proposal on collaborative cross entropy to solve combinatorial optimization problems [15]. In addition to the above method, Lu et al. used a KNN combination of distance thresholds to determine the weight [16]. And other scholars used algorithm combinations to determine the weight [17–20]. In recent years, ensemble learning has become a research hotspot, and some scholars have determined the contribution degree of attributes to classification results through ensemble learning algorithms, for example, random forest [21], XGBoost, etc. Random forest determines the weight by calculating the attribute contribution, which is a way of calculating the weight value developed with the development of ensemble learning [22]. And Liu et al. constructed multiple mixed 0–1 linear programming models (MLPMs) to obtain the classification range of alternatives and weights of policy attributes applied in maldistributed decision-making problems [23].In this paper, a distance-based classification algorithm is proposed to find the minimum distance between the midpoint of the category to which the data belong and the attribute vector. The distance between data points in the same category is closer and the distance between data points in different categories is farther to achieve the effect of improving the classification. In this paper, Lingo is used to solve the weights, and the solved weights are applied to the K-means clustering iris data set, wine data set, and wheat seed data set. Compared with the weights determined by the class and entropy weight method, the method proposed in this paper has different degrees of improvement in the clustering effect.The key contributions of this work are as follows: (1) The algorithm accurately determines the attribute weights and identifies the solution from the data set itself. (2) This method overcomes the shortcomings of AHP and other methods. (3) It is less subjective and does not need to calculate entropy [24, 25]. (4) There is no need to use formulas such as variance to obtain attribute weights. There is no need for many trial and error steps, and there is no need for integrated learning to build models.The rest of this paper is organized as follows: Section2 explains the idea of solving the weights in this paper. Section 3 describes the K-means clustering process and evaluation indicators. Section 4 describes the experimental procedure. Section 5 is a summary of the full text.


## 2. Determining Weights

### 2.1. The Solution Idea

The purpose of clustering and classification is to obtain groups such that objects within a group are more similar than objects in different groups [26]. The weights are determined by minimizing the distances between attribute vectors within the same group and the center vector to maximize the distance between the different groups, thus effectively separating the different clusters. When the distance between the attribute vectors of each group and the center of the group reaches the minimum value, the distance between the different groups is maximized. The weight determined is the optimal attribute weight. The weight of the solution is applied to a known or unknown data set to improve the learning effect. The solution idea comes from the KNN algorithm [27].


#### 2.1.1. KNN Algorithm

The KNN algorithm is a relatively mature and simple machine learning algorithm in theory. The idea of KNN is that if a sample has a high probability of belonging to a certain category among thek nearest samples in the feature space, and most of them belong to a certain category, then the sample is also classified in this category. KNN is classified by measuring the distance between different characteristic values, generally using the Euclidean distance. In classification decisions, this method only determines the category of the samples to be classified according to the category of the nearest sample or several samples. The KNN solution process is as follows:Step 1: Calculate distances. The distance between characteristic values is calculated, the distance between the test data and each training data value. Generally, the Euclidean distance is used for calculation, and the Manhattan distance and Mahalanobis distance can also be used. Table1 shows some distance formulas.Step 2: Sort by increasing distance.Step 3: Classify samples according to distance. Selectk data points with the smallest distance from the sample point to determine the type of data with the highest frequency among the K sample points.Step 4: Identify categories. The category with the highest frequency in the firstK points is used as the predictive classification of the test data. Classification methods are divided into simple and weighted voting methods.Table 1
Several commonly used distance formulas.
Distance nameBrief explanationDistance formulaEuclidean distanceThe straight-line distance between two pointsd=∑k=1nx1k−x2k2Manhattan distanceThe sum of the absolute wheelbases of two points in standard coordinatesd=∑k=1nx1k−x2kChebyshev distanceThe maximum value of the difference between coordinatesd=maxix1i−x2iMarkov distanceThe covariance distance of datad=X−μTS−1X−μ. The covariance matrix is denoted as S, and the mean is denoted as μ


#### 2.1.2. Weight Solution Idea

The idea of solving weights comes from the reverse solution method of KNN. KNN makes classification judgments according to the occurrence frequency of categories, and the purpose of determining the weight is to improve the learning effect. In the KNN algorithm, we aim to make allk surrounding sample points belong to a certain category. The distance between samples of the same category should be small, and the distance between samples of different categories should be large. The minimum distance between the sample vector of a category and the center point is reflected in the sample vector of the category. The steps of determining the weights are as follows:Step 1: Identify categories. Classify sample data of different categories according to the known data.Step 2: ChooseK. The sample number of each category is calculated after classification, and the value K is the sample number of the category.Step 3: Calculate the distance. Calculate the distance between the sample of the category and the center point vector, carry out the weighting calculation, and obtain the weight when the distance is the smallest.


### 2.2. Solution Process

The goal of this method is to minimize the distance between a classification sample of the data set and the center point of the category to which it belongs. In this experiment, the Euclidean distance is adopted. In addition to the Euclidean distance, other distance functions, such as the Mahalanobis distance, Manhattan distance, and Chebyshev distance, can be adopted. This paper presents an accurate analytical method for weighted attribute distance functions.Sample classificationC=c1⋮ci⋮cn. The attribute vector of each sample is xi=r1i⋮rpi⋮rki. The attribute vector values of the center point under the label are si=s1i⋮spi⋮skii=1,2,3,…,n, where λpp=1,2,…,k is the weight of each attribute. The constraint conditions are 0≤λp≤1, ∑p=1kλp=1, and n=n1+n2+⋯+ni+⋯+nn.Let us define the objective function as(1)sd=arg min∑m=1n∑i=1ni∑p=1kλp2rpi−spi2m,where ni is the number of samples under each category and n is the total number of samples. By solving the attribute vector of the center point of each label, the minimum value λp of the objective function is obtained by taking the partial derivative or using the gradient descent method. When sd is the minimum value, the weight λp of each attribute is obtained. Namely, the sum of the distance between the sample point of each category and the center point of each category is the smallest. Table 2 shows the meanings of the other parameters. In this experiment, the Euclidean distance is used to determine the weight; other distances can also be used for the calculation.Table 2
Brief explanations of various parameters.
Parameter nameParameter meaningCSample classificationxiSample attribute vectorsiVector of the center point under each classλpWeight valueniNumber of samples under the categorynTotal number of sampleskNumber of attributessdObjective function value


## 2.1. The Solution Idea

The purpose of clustering and classification is to obtain groups such that objects within a group are more similar than objects in different groups [26]. The weights are determined by minimizing the distances between attribute vectors within the same group and the center vector to maximize the distance between the different groups, thus effectively separating the different clusters. When the distance between the attribute vectors of each group and the center of the group reaches the minimum value, the distance between the different groups is maximized. The weight determined is the optimal attribute weight. The weight of the solution is applied to a known or unknown data set to improve the learning effect. The solution idea comes from the KNN algorithm [27].


### 2.1.1. KNN Algorithm

The KNN algorithm is a relatively mature and simple machine learning algorithm in theory. The idea of KNN is that if a sample has a high probability of belonging to a certain category among thek nearest samples in the feature space, and most of them belong to a certain category, then the sample is also classified in this category. KNN is classified by measuring the distance between different characteristic values, generally using the Euclidean distance. In classification decisions, this method only determines the category of the samples to be classified according to the category of the nearest sample or several samples. The KNN solution process is as follows:Step 1: Calculate distances. The distance between characteristic values is calculated, the distance between the test data and each training data value. Generally, the Euclidean distance is used for calculation, and the Manhattan distance and Mahalanobis distance can also be used. Table1 shows some distance formulas.Step 2: Sort by increasing distance.Step 3: Classify samples according to distance. Selectk data points with the smallest distance from the sample point to determine the type of data with the highest frequency among the K sample points.Step 4: Identify categories. The category with the highest frequency in the firstK points is used as the predictive classification of the test data. Classification methods are divided into simple and weighted voting methods.Table 1
Several commonly used distance formulas.
Distance nameBrief explanationDistance formulaEuclidean distanceThe straight-line distance between two pointsd=∑k=1nx1k−x2k2Manhattan distanceThe sum of the absolute wheelbases of two points in standard coordinatesd=∑k=1nx1k−x2kChebyshev distanceThe maximum value of the difference between coordinatesd=maxix1i−x2iMarkov distanceThe covariance distance of datad=X−μTS−1X−μ. The covariance matrix is denoted as S, and the mean is denoted as μ


### 2.1.2. Weight Solution Idea

The idea of solving weights comes from the reverse solution method of KNN. KNN makes classification judgments according to the occurrence frequency of categories, and the purpose of determining the weight is to improve the learning effect. In the KNN algorithm, we aim to make allk surrounding sample points belong to a certain category. The distance between samples of the same category should be small, and the distance between samples of different categories should be large. The minimum distance between the sample vector of a category and the center point is reflected in the sample vector of the category. The steps of determining the weights are as follows:Step 1: Identify categories. Classify sample data of different categories according to the known data.Step 2: ChooseK. The sample number of each category is calculated after classification, and the value K is the sample number of the category.Step 3: Calculate the distance. Calculate the distance between the sample of the category and the center point vector, carry out the weighting calculation, and obtain the weight when the distance is the smallest.


## 2.1.1. KNN Algorithm

The KNN algorithm is a relatively mature and simple machine learning algorithm in theory. The idea of KNN is that if a sample has a high probability of belonging to a certain category among thek nearest samples in the feature space, and most of them belong to a certain category, then the sample is also classified in this category. KNN is classified by measuring the distance between different characteristic values, generally using the Euclidean distance. In classification decisions, this method only determines the category of the samples to be classified according to the category of the nearest sample or several samples. The KNN solution process is as follows:Step 1: Calculate distances. The distance between characteristic values is calculated, the distance between the test data and each training data value. Generally, the Euclidean distance is used for calculation, and the Manhattan distance and Mahalanobis distance can also be used. Table1 shows some distance formulas.Step 2: Sort by increasing distance.Step 3: Classify samples according to distance. Selectk data points with the smallest distance from the sample point to determine the type of data with the highest frequency among the K sample points.Step 4: Identify categories. The category with the highest frequency in the firstK points is used as the predictive classification of the test data. Classification methods are divided into simple and weighted voting methods.Table 1
Several commonly used distance formulas.
Distance nameBrief explanationDistance formulaEuclidean distanceThe straight-line distance between two pointsd=∑k=1nx1k−x2k2Manhattan distanceThe sum of the absolute wheelbases of two points in standard coordinatesd=∑k=1nx1k−x2kChebyshev distanceThe maximum value of the difference between coordinatesd=maxix1i−x2iMarkov distanceThe covariance distance of datad=X−μTS−1X−μ. The covariance matrix is denoted as S, and the mean is denoted as μ


## 2.1.2. Weight Solution Idea

The idea of solving weights comes from the reverse solution method of KNN. KNN makes classification judgments according to the occurrence frequency of categories, and the purpose of determining the weight is to improve the learning effect. In the KNN algorithm, we aim to make allk surrounding sample points belong to a certain category. The distance between samples of the same category should be small, and the distance between samples of different categories should be large. The minimum distance between the sample vector of a category and the center point is reflected in the sample vector of the category. The steps of determining the weights are as follows:Step 1: Identify categories. Classify sample data of different categories according to the known data.Step 2: ChooseK. The sample number of each category is calculated after classification, and the value K is the sample number of the category.Step 3: Calculate the distance. Calculate the distance between the sample of the category and the center point vector, carry out the weighting calculation, and obtain the weight when the distance is the smallest.


## 2.2. Solution Process

The goal of this method is to minimize the distance between a classification sample of the data set and the center point of the category to which it belongs. In this experiment, the Euclidean distance is adopted. In addition to the Euclidean distance, other distance functions, such as the Mahalanobis distance, Manhattan distance, and Chebyshev distance, can be adopted. This paper presents an accurate analytical method for weighted attribute distance functions.Sample classificationC=c1⋮ci⋮cn. The attribute vector of each sample is xi=r1i⋮rpi⋮rki. The attribute vector values of the center point under the label are si=s1i⋮spi⋮skii=1,2,3,…,n, where λpp=1,2,…,k is the weight of each attribute. The constraint conditions are 0≤λp≤1, ∑p=1kλp=1, and n=n1+n2+⋯+ni+⋯+nn.Let us define the objective function as(1)sd=arg min∑m=1n∑i=1ni∑p=1kλp2rpi−spi2m,where ni is the number of samples under each category and n is the total number of samples. By solving the attribute vector of the center point of each label, the minimum value λp of the objective function is obtained by taking the partial derivative or using the gradient descent method. When sd is the minimum value, the weight λp of each attribute is obtained. Namely, the sum of the distance between the sample point of each category and the center point of each category is the smallest. Table 2 shows the meanings of the other parameters. In this experiment, the Euclidean distance is used to determine the weight; other distances can also be used for the calculation.Table 2
Brief explanations of various parameters.
Parameter nameParameter meaningCSample classificationxiSample attribute vectorsiVector of the center point under each classλpWeight valueniNumber of samples under the categorynTotal number of sampleskNumber of attributessdObjective function value


## 3. K-Means Algorithm

### 3.1. K-Means Algorithm Process

The K-means algorithm is an unsupervised learning algorithm that has become one of the most widely used clustering algorithms [28, 29]. It is a distance-based clustering algorithm that uses the distance between objects as an evaluation index of similarity.The traditional K-means Algorithm 1 process is as follows:Algorithm 1: K-means clustering process.
Input: number of clustersK, data set DOutput:K clusters.Algorithm steps:Step 1: TakeK, which means we will divide the data set into K groups.Step 2: Randomly selectK points from the data set as the initial clustering centers.Step 3: Calculate the distances between all points and theK cluster centers and put the samples into the class with the center with the shortest distance.Step 4: Calculate the average coordinates of the data points in each class cluster to update the center of the cluster.Step 5: Repeat steps (3) and (4) until the cluster center remains unchanged.


### 3.2. Evaluation Indicators

In this experiment, the normalized mutual information [30, 31] (NMI) is used as the evaluation index of clustering quality. NMI is commonly used in clustering to measure the similarity of two clustering results. It can objectively evaluate the accuracy of an algorithm partition compared with the standard partition. The range of NMI is 0 to 1, and the higher it is, the greater the accuracy is. The concept of NMI comes from relative entropy, namely, KL divergence and mutual information.Relative entropy is an asymmetrical measure of the difference between two probability distributions, and in the discrete case, it is defined as(2)KLp||q=∑pxlogpxqx,where p(x) and q(x) are the two probability distributions of the random variable x.Mutual information [32] is a useful information measure in information theory. It can be regarded as the amount of information contained in a random variable about another random variable. Mutual information is the relative entropy of the joint probability distribution and edge probability product distribution of two random variables X and Y, which is defined as(3)IX;Y=∑x∑ypx,ylogpx,ypxpy.Normalized mutual information is the result of the normalization of mutual information and is defined as(4)NMIX;Y=2IX;YHX+HY,where HX and HY are the information entropy of the random variables X and Y and IX;Y is the mutual information of X and Y.


### 3.3. K-Means with the Accurate Weight Determination Method

The traditional K-means algorithm does not consider the importance degree of attributes, so the distance weights from each attribute to the center point of the cluster are equal. However, in many cases, the importance of different attributes may not be equal. Application of traditional K-means to these scenarios will inevitably lead to inaccurate clustering results. In this paper, the exact solution process of feature weights is carried out before the K-means algorithm is applied. The obtained weights are weighted by the distance between each attribute and the center point to obtain the final distance between the sample point and the center of the cluster. Figure1 shows the flowchart of the k-means algorithm using the exact weight solution method.Figure 1
Flowchart of applying the exact weight determination method to the K-means algorithm.


## 3.1. K-Means Algorithm Process

The K-means algorithm is an unsupervised learning algorithm that has become one of the most widely used clustering algorithms [28, 29]. It is a distance-based clustering algorithm that uses the distance between objects as an evaluation index of similarity.The traditional K-means Algorithm 1 process is as follows:Algorithm 1: K-means clustering process.
Input: number of clustersK, data set DOutput:K clusters.Algorithm steps:Step 1: TakeK, which means we will divide the data set into K groups.Step 2: Randomly selectK points from the data set as the initial clustering centers.Step 3: Calculate the distances between all points and theK cluster centers and put the samples into the class with the center with the shortest distance.Step 4: Calculate the average coordinates of the data points in each class cluster to update the center of the cluster.Step 5: Repeat steps (3) and (4) until the cluster center remains unchanged.


## 3.2. Evaluation Indicators

In this experiment, the normalized mutual information [30, 31] (NMI) is used as the evaluation index of clustering quality. NMI is commonly used in clustering to measure the similarity of two clustering results. It can objectively evaluate the accuracy of an algorithm partition compared with the standard partition. The range of NMI is 0 to 1, and the higher it is, the greater the accuracy is. The concept of NMI comes from relative entropy, namely, KL divergence and mutual information.Relative entropy is an asymmetrical measure of the difference between two probability distributions, and in the discrete case, it is defined as(2)KLp||q=∑pxlogpxqx,where p(x) and q(x) are the two probability distributions of the random variable x.Mutual information [32] is a useful information measure in information theory. It can be regarded as the amount of information contained in a random variable about another random variable. Mutual information is the relative entropy of the joint probability distribution and edge probability product distribution of two random variables X and Y, which is defined as(3)IX;Y=∑x∑ypx,ylogpx,ypxpy.Normalized mutual information is the result of the normalization of mutual information and is defined as(4)NMIX;Y=2IX;YHX+HY,where HX and HY are the information entropy of the random variables X and Y and IX;Y is the mutual information of X and Y.


## 3.3. K-Means with the Accurate Weight Determination Method

The traditional K-means algorithm does not consider the importance degree of attributes, so the distance weights from each attribute to the center point of the cluster are equal. However, in many cases, the importance of different attributes may not be equal. Application of traditional K-means to these scenarios will inevitably lead to inaccurate clustering results. In this paper, the exact solution process of feature weights is carried out before the K-means algorithm is applied. The obtained weights are weighted by the distance between each attribute and the center point to obtain the final distance between the sample point and the center of the cluster. Figure1 shows the flowchart of the k-means algorithm using the exact weight solution method.Figure 1
Flowchart of applying the exact weight determination method to the K-means algorithm.


## 4. Experimental Process

### 4.1. Introduction to the Data Sets

#### 4.1.1. Iris Data Set

The iris data set is a commonly used machine learning data set [33]. It includes four attributes, the length of the calyx (Speal Length), the width of the calyx (Speal Width), the length of the petal (Petal Length), and the width of the petal (Petal Width). The unit of the four attributes is CM, which is a numerical variable, and there are no missing values. Figure 2 shows a scatter plot of iris data attributes. Figure 3 shows the histogram of iris data attributes. The mountain iris, chameleon iris, and Virginia iris are the three categories. Each category collects 50 sample records, for a total of 150 irises.Figure 2
Scatter plot of calyx length and width and petal length and width in the iris data set.Figure 3
Histogram and scatter diagram of each attribute in the iris data set.


#### 4.1.2. Wine Data Set

The wine data set is a publicly available data set from the University of California Irvine (UCI). It is the result of a chemical analysis of wines grown in the same region of Italy from three different varieties. The analysis determined the values of 13 attributes of each of the three wines. The attributes are class identifiers, represented by categories 1, 2, and 3. Figure4 shows the distribution of wine attributes. There are 59 samples in category 1, 71 samples in category 2, and 48 samples in category 3. There are no missing values in this data set.Figure 4
Wine data set.


#### 4.1.3. Wheat Seed Data Set

The wheat seed data set is commonly used in classification and clustering tasks. There are 210 records, 7 features, and 1 label in the data set. Figure5 shows the distribution of wheat seed attributes. The labels are divided into 3 categories with 70 samples in each category, and there are no missing values.Figure 5
Wheat seed data set.


### 4.2. Determining Attribute Weights

#### 4.2.1. Determining the Attribute Weights of the Iris Data Set

The category number of the iris data set is 3, so the objective function used to determine the weights of the four attributes according to formula (1) is(5)sd=arg min∑i=1n1∑p=1kλp2rpi−sph2+∑j=1n2∑p=1kλp2rpj−spl2+∑r=1n3∑p=1kλp2rps−spm2,where n1,n2,n3 are the numbers of samples of mountain iris, chameleon iris, and Virginia iris, respectively; n is the total number of samples; k is the number of attributes; the iris has four attributes of calyx length, calyx width, petal length, and petal width (so k = 4); and the meanings of the other parameters are given below. Table 3 illustrates the number of irises and the k value for each category, and Table 4 shows the center vectors and parameter meanings of various types of flowers.Table 3
Main parameter explanation and the determined values of the objective functions of the iris attribute weights.
SymbolBrief explanationNumerical valuen1Number of mountain iris samples50n2Number of chameleon iris samples50n3Number of Virginia iris samples50kNumber of data set attributes4Table 4
Explanation of other parameters used in solving the objective function of iris attribute weights.
SymbolBrief explanationshCenter vector of mountain iris samplesslCenter vector of chameleon iris samplessmCenter vector of Virginia iris samplesxiSample attribute vector of the categoryWe separate the three categories of the data set and calculate the attribute vector values of the center points under the three tagssh=5.0063.4281.4620.246, sl=5.9362.774.261.326, and sm=6.5882.9745.5522.026. In the experiment, LINGO12.0 is used to solve, and the values are rounded to λkk=1,2,3,4 in Table 5.Table 5
The weight values of the iris characteristic attributes are accurately determined.
WeightValueλ10.053λ20.117λ30.107λ40.722


#### 4.2.2. Determining the Wine Data Set Attribute Weights

The number of sample categories in the wine data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, and K is the number of attributes. The meaning of each parameter is given below. Table 6 lists the number and parameter significance of the three categories of the wine data set. Table 7 illustrates the three categories of wine center vector parameters.Table 6
Main parameters and values of the objective functions of wine attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 159n2Samples with a category of 271n3Samples with a category of 348kNumber of data set attributes13Table 7
Explanation of the other parameters of the objective functions of wine attribute weights.
SymbolBrief explanationshThe sample category has 1 center vectorslThe sample category has 2 center vectorssmThe sample category has 3 center vectorsxiThe sample attribute vector of the categoryThe vector values of the attributes of the center points under the three labels aresh=1.057,0.860,1.037,0.873,1.066,1.237,1.469,0.801,1.193,1.092,1.109,1.209,1.493T, sl=0.9444,0.827,0.948,1.038,0.947,0.984,1.025,1.004,1.024,0.610,1.103,1.066,0.695T, and sm=1.011,1.426,1.029,1.098,0.995,0.731,0.385,1.236,0.725,1.462,0.713,0.644,0.843T.The rounded resultsλ3λkk=1,2,…,13 are given below. Table 8 shows the weight values.Table 8
The weight values of the wine characteristic attributes are accurately obtained.
WeightValueλ10.745λ20.0043λ30.065λ40.036λ50.036λ60.017λ70.01λ80.0068λ90.0077λ100.0069λ110.023λ120.026λ130.011


#### 4.2.3. Determining the Attribute Weights of the Wheat and Wheat Seed Data Set

The number of sample categories in the wheat seed data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, k is the number of attributes, and the meanings of each parameter are as given below. Table 9 shows the parameter values needed to calculate the weight of wheat seeds.Table 9
Main parameter explanation and value of the objective functions in determining wheat seed attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 170n2Samples with a category of 270n3Samples with a category of 370kNumber of data set attributes7The meanings of the other attributes are the same as in Table7. The vector values of the attributes of the center point under the three labels are sh=0.965,0.981,1.010,0.978,0.995,0.720,0.940T, sl=1.234,1.108,1.014,1.092,1.128,0.985,1.113T, and sm=0.799,0.909,0.975,0.929,0.875,1.294,0.946T. The rounded λkk=1,2…7 results are given below. Table 10 shows the calculated weights of wheat seed attributes.Table 10
Weight values of wheat seed characteristic attributes obtained by the accurate solution method.
WeightValueλ10.0328λ20.151λ30.504λ40.133λ50.071λ60.0012λ70.104


### 4.3. Analysis of the Experimental Results

The methods of K-means with accurately determined weights, traditional K-means, and K-means with entropy weights are used to cluster the iris, wine, and wheat seed data sets. The normalized mutual information and the confusion matrix [34] are used as evaluation criteria to evaluate the three methods.


#### 4.3.1. Weight Entropy Method

The basic idea of the entropy weight method [35, 36] used to determine the objective weight is the index variability. Weight is determined according to the information entropy [37], which is the expectation of information content. The probability of the occurrence of a data value is negatively correlated with it. The higher the information entropy of an attribute is, the less information it can provide, the smaller the role it plays in evaluation, and the smaller its weight is. Table 11 shows the weight values of iris attributes obtained by the exact solution method. Table 12 shows the weight values of the attributes of the wine data set obtained by the exact solution method. Table 13 shows the weight values of the attributes of the wheat seed data set obtained by the exact solution method.Table 11
Weight values of iris attributes obtained by the entropy weight method.
WeightValueλ10.193λ20.112λ30.318λ40.376Table 12
Weight values of wine characteristic attributes obtained by the entropy weight method.
WeightValueλ10.049λ20.123λ30.022λ40.041λ50.059λ60.066λ70.109λ80.080λ90.067λ100.099λ110.069λ120.091λ130.120Table 13
Weight values of wheat seed characteristic attributes obtained by the entropy weight method.
WeightValueλ10.205λ20.158λ30.07λ40.155λ50.168λ60.115λ70.126


#### 4.3.2. Iris Data Clustering Results

The experiment is implemented in the Python 3.8.5 environment, and the maximum number of K-means iterations after inputting the attribute weight is 200. The normalized mutual information is selected as the evaluation criterion, and the confusion matrix is established. The normalized mutual information can make the clustering results to 0-1 so that the clustering accuracy of the two methods can be seen intuitively [38]. The effect of clustering on a certain category can be obtained through a confusion matrix [39]. Clustering results can be visualized to make the results more intuitive [40]. The above methods are used to compare the results of the K-means algorithm with weights, K-means without weights, and K-means with weights determined by the entropy weight method. Table 14 shows the NMI of the iris data set after clustering by the three methods.Table 14
NMI of the iris data set after clustering by the three methods.
Normalized mutual information (NMI)K-means with the exact weight0.864K-means without weight0.758K-means with the entropy weight0.785NMI is an external evaluation standard method for clustering [41]. By calculating the normalized mutual information of the real labels and the labels after clustering, the accuracy of clustering can be seen [42, 43]. The NMI of the three methods after clustering the iris data set is shown in Table 14. First, it can be concluded from the table that the NMI after clustering by K-means with weights is approximately 0.11 higher than that after clustering without weights. The clustering effect of K-means after determining the attribute weights is better, which confirms the feasibility of this method. Second, when the results obtained by the entropy weight method are put into the K-means algorithm, the NMI after clustering is 0.785. It is 0.03 higher than that of traditional K-means without weights. However, the NMI after clustering of the algorithm proposed in this paper for accurately determining the weights is 0.08 higher than that of the entropy weight method. Finally, although the weight determined by the entropy weight method improves the accuracy of the iris data clustering class to a certain extent compared with clustering without weights, it is far from the improvement achieved by the weight determination method proposed in this paper. The confusion matrix after clustering is given below. Table 15 shows the confusion matrix of the effect of the three methods on iris clustering.Table 15
Confusion matrix of the three methods for clustering the iris data set.
Confusion matrixMountain irisChameleon irisVirginia irisAccurate methodReal category yMountain iris5000Chameleon iris0482Virginia iris0446Without weightMountain iris5000Chameleon iris0482Virginia iris01436Entropy weightsMountain iris5000Chameleon iris0491Virginia iris01337The confusion matrix is an effective tool for evaluating classifications and clustering criteria [44], as it can be used to clearly see in which categories the model does not perform well [45]. The confusion matrix [46, 47] after the three methods of clustering is shown in Table 15. First, it can be seen from the table that the clustering effect of the three methods is equally good for the mountain iris. These samples can be clustered accurately. All three methods are largely accurate in the category of the chameleon iris, but there is a large difference among the three in the category of the Virginia iris. K-means without weights incorrectly clustered 14 samples of Virginia iris into the category of chameleon iris. Compared with K-means clustering without weights, the improvement of K-means clustering after weight determination by the entropy weight method is not very large. Second, for the clustering of Virginia iris, the weight clustering results are almost the same as those of all attributes after weight determination by the entropy weight method. After the weights are determined by the entropy weight method, 13 Virginia irises are incorrectly clustered into the chameleon iris category, while only 14 samples are incorrectly classified even with uncertain weights. Neither method could accurately cluster Virginia irises, and it was more difficult to cluster Virginia irises than the other two iris categories. Figure 2 shows that the calyx and petal lengths and widths of the Virginia iris and chameleon iris are similar. The data are mixed and difficult to distinguish, which means that the two methods cannot distinguish the two flower categories well. The difference in the properties of the mountain iris and the other two flowers is relatively large. The weight obtained by the algorithm with the accurate solution is applied to K-means, which can distinguish the two categories well, proving the accuracy and efficiency of the method. Finally, the effect of K-means clustering determined by the entropy weight method is visualized.Figure6 shows the results of clustering the iris data set with the weights obtained by our method. Figure 7 shows the clustering results without attribute weights. Figure 8 shows the clustering results of the weights determined by the entropy weight method. The effect diagram after clustering shows more intuitively that some sample points are still mixed in the clustering results of chameleon iris and Virginia iris by K-means without weights. These points are not effectively divided into different clusters. However, K-means with accurately determined weights has a better effect on the clustering of the two types. Points of different categories are effectively clustered into different clusters.Figure 6
K-means clustering effect diagram of the iris data after the exact solution method is used to determine the weights.Figure 7
Effect diagram of K-means without weights on iris data clustering.Figure 8
K-means clustering effect diagram of iris data after the entropy weight method is used to determine the weights.


#### 4.3.3. Wine Data Clustering Results

The wine data set has more attributes than the iris data set. The results of the following three methods are compared: the K-means algorithm for calculating the weights by the exact solution method, K-means without weights, and K-means with weights determined by the entropy method. Table16 shows the NMI values of the three methods for clustering the wine data set, and Table 17 shows the confusion matrix of the three methods for clustering the wine data set.Table 16
NMI aggregated by the three methods for wine data.
Normalized mutual information (NMI)K-means with the exact weight0.865K-means without weights0.765K-means with entropy weight0.765Table 17
Confusion matrix of the three methods for clustering the wine data set.
Confusion matrixCategory y_pred after K-means clustering with the exact solution methodCategory 1Category 2Category 3Accurate methodReal category yCategory 15900Category 25642Category 30048Without weightCategory 15801Category 22609Category 31047Entropy weightsCategory 15810Category 22609Category 30147According to the NMI after clustering by the three methods, the method for solving the weight proposed in this paper improves the results by approximately 0.1 compared with those of the other two methods. The entropy weight method does not improve the results much in the wine data clustering class, so different weight solving methods apply to different situations. According to the confusion matrix after clustering by the three methods, the exact solution method performs better than the other two methods on the three sample categories. There is little difference between the entropy weight method and K-means without weights. Figure9 shows the clustering results of the wine data set by the method in this paper, Figure 10 shows the clustering results without attribute weights, and Figure 11 shows the entropy weighting method clustering results.Figure 9
Effect diagram of K-means clustering of the wine data after the exact solution method is used to determine the weights.Figure 10
Effect diagram of K-means without weights on wine data clustering.Figure 11
Effect diagram of K-means clustering of the wine data after the entropy weight method is used to determine the weights.


#### 4.3.4. Cluster Results on Wheat Seed Data

The number of attributes in the wheat seed data set is between those of the iris data set and the wine data set. The results of the weighted K-means algorithm, the K-means package in SKLearn, and weighted K-means with weights determined by the entropy weight method are compared below. Table18 shows the NMI results of the three methods for clustering wheat seeds, and Table 19 shows the confusion matrix results after clustering.Table 18
NMI of wheat seed data aggregated by the three methods after classification.
Normalized mutual information (NMI)K-means with the exact weight0.673K-means without weights0.524K-means with entropy weight0.621Table 19
Confusion matrix of the three methods for clustering the wheat seed data set.
Confusion matrixCategory 1Category 2Category 3Accurate methodReal category yCategory 155213Category 29610Category 31069Without weightCategory 158111Category 210600Category 320050Entropy weightsCategory 160100Category 212579Category 33265The attribute importance of the wheat seed data set varies. Compared with the K-means clustering results without weights, the normalized mutual information after K-means clustering with weights is greatly improved. The normalized mutual information after applying the exact solution method and the entropy weight method of determining the weights is improved by 0.15 and 0.1, respectively. However, compared with the entropy weight method, the weight method proposed in this paper improves the normalized mutual information by 0.05, and the clustering effect is better.It can be seen from the confusion matrix after clustering by the three methods that the improvement of weighted K-means compared with unweighted K-means is mainly in the data set of categories 3. The number of correct samples in the clustering of the precise solution method and entropy weight method is increased by 19 and 15, respectively, compared with that of traditional K-means. Compared with the entropy weight method, the exact solution method performs better in the clustering of category 3. Figure12 represents the clustering result of wheat seed data by the method in this paper, while Figure 13 shows the clustering results of the unweighted data set. Figure 14 shows the clustering results of the wheat seed data set by the entropy weight method. As seen from the visualization, the clustering effects of the accurate solution method and entropy weight method are significantly better than that of traditional K-means. Samples of different categories are divided into different clusters.Figure 12
K-means clustering effect diagram of wheat seed data after the exact solution method is used to determine the weights.Figure 13
Effect diagram of K-means without weights on wheat seed data clustering.Figure 14
The K-means clustering effect of wheat seed data after the entropy weight method is used to determine the weights.


## 4.1. Introduction to the Data Sets

### 4.1.1. Iris Data Set

The iris data set is a commonly used machine learning data set [33]. It includes four attributes, the length of the calyx (Speal Length), the width of the calyx (Speal Width), the length of the petal (Petal Length), and the width of the petal (Petal Width). The unit of the four attributes is CM, which is a numerical variable, and there are no missing values. Figure 2 shows a scatter plot of iris data attributes. Figure 3 shows the histogram of iris data attributes. The mountain iris, chameleon iris, and Virginia iris are the three categories. Each category collects 50 sample records, for a total of 150 irises.Figure 2
Scatter plot of calyx length and width and petal length and width in the iris data set.Figure 3
Histogram and scatter diagram of each attribute in the iris data set.


### 4.1.2. Wine Data Set

The wine data set is a publicly available data set from the University of California Irvine (UCI). It is the result of a chemical analysis of wines grown in the same region of Italy from three different varieties. The analysis determined the values of 13 attributes of each of the three wines. The attributes are class identifiers, represented by categories 1, 2, and 3. Figure4 shows the distribution of wine attributes. There are 59 samples in category 1, 71 samples in category 2, and 48 samples in category 3. There are no missing values in this data set.Figure 4
Wine data set.


### 4.1.3. Wheat Seed Data Set

The wheat seed data set is commonly used in classification and clustering tasks. There are 210 records, 7 features, and 1 label in the data set. Figure5 shows the distribution of wheat seed attributes. The labels are divided into 3 categories with 70 samples in each category, and there are no missing values.Figure 5
Wheat seed data set.


## 4.1.1. Iris Data Set

The iris data set is a commonly used machine learning data set [33]. It includes four attributes, the length of the calyx (Speal Length), the width of the calyx (Speal Width), the length of the petal (Petal Length), and the width of the petal (Petal Width). The unit of the four attributes is CM, which is a numerical variable, and there are no missing values. Figure 2 shows a scatter plot of iris data attributes. Figure 3 shows the histogram of iris data attributes. The mountain iris, chameleon iris, and Virginia iris are the three categories. Each category collects 50 sample records, for a total of 150 irises.Figure 2
Scatter plot of calyx length and width and petal length and width in the iris data set.Figure 3
Histogram and scatter diagram of each attribute in the iris data set.


## 4.1.2. Wine Data Set

The wine data set is a publicly available data set from the University of California Irvine (UCI). It is the result of a chemical analysis of wines grown in the same region of Italy from three different varieties. The analysis determined the values of 13 attributes of each of the three wines. The attributes are class identifiers, represented by categories 1, 2, and 3. Figure4 shows the distribution of wine attributes. There are 59 samples in category 1, 71 samples in category 2, and 48 samples in category 3. There are no missing values in this data set.Figure 4
Wine data set.


## 4.1.3. Wheat Seed Data Set

The wheat seed data set is commonly used in classification and clustering tasks. There are 210 records, 7 features, and 1 label in the data set. Figure5 shows the distribution of wheat seed attributes. The labels are divided into 3 categories with 70 samples in each category, and there are no missing values.Figure 5
Wheat seed data set.


## 4.2. Determining Attribute Weights

### 4.2.1. Determining the Attribute Weights of the Iris Data Set

The category number of the iris data set is 3, so the objective function used to determine the weights of the four attributes according to formula (1) is(5)sd=arg min∑i=1n1∑p=1kλp2rpi−sph2+∑j=1n2∑p=1kλp2rpj−spl2+∑r=1n3∑p=1kλp2rps−spm2,where n1,n2,n3 are the numbers of samples of mountain iris, chameleon iris, and Virginia iris, respectively; n is the total number of samples; k is the number of attributes; the iris has four attributes of calyx length, calyx width, petal length, and petal width (so k = 4); and the meanings of the other parameters are given below. Table 3 illustrates the number of irises and the k value for each category, and Table 4 shows the center vectors and parameter meanings of various types of flowers.Table 3
Main parameter explanation and the determined values of the objective functions of the iris attribute weights.
SymbolBrief explanationNumerical valuen1Number of mountain iris samples50n2Number of chameleon iris samples50n3Number of Virginia iris samples50kNumber of data set attributes4Table 4
Explanation of other parameters used in solving the objective function of iris attribute weights.
SymbolBrief explanationshCenter vector of mountain iris samplesslCenter vector of chameleon iris samplessmCenter vector of Virginia iris samplesxiSample attribute vector of the categoryWe separate the three categories of the data set and calculate the attribute vector values of the center points under the three tagssh=5.0063.4281.4620.246, sl=5.9362.774.261.326, and sm=6.5882.9745.5522.026. In the experiment, LINGO12.0 is used to solve, and the values are rounded to λkk=1,2,3,4 in Table 5.Table 5
The weight values of the iris characteristic attributes are accurately determined.
WeightValueλ10.053λ20.117λ30.107λ40.722


### 4.2.2. Determining the Wine Data Set Attribute Weights

The number of sample categories in the wine data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, and K is the number of attributes. The meaning of each parameter is given below. Table 6 lists the number and parameter significance of the three categories of the wine data set. Table 7 illustrates the three categories of wine center vector parameters.Table 6
Main parameters and values of the objective functions of wine attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 159n2Samples with a category of 271n3Samples with a category of 348kNumber of data set attributes13Table 7
Explanation of the other parameters of the objective functions of wine attribute weights.
SymbolBrief explanationshThe sample category has 1 center vectorslThe sample category has 2 center vectorssmThe sample category has 3 center vectorsxiThe sample attribute vector of the categoryThe vector values of the attributes of the center points under the three labels aresh=1.057,0.860,1.037,0.873,1.066,1.237,1.469,0.801,1.193,1.092,1.109,1.209,1.493T, sl=0.9444,0.827,0.948,1.038,0.947,0.984,1.025,1.004,1.024,0.610,1.103,1.066,0.695T, and sm=1.011,1.426,1.029,1.098,0.995,0.731,0.385,1.236,0.725,1.462,0.713,0.644,0.843T.The rounded resultsλ3λkk=1,2,…,13 are given below. Table 8 shows the weight values.Table 8
The weight values of the wine characteristic attributes are accurately obtained.
WeightValueλ10.745λ20.0043λ30.065λ40.036λ50.036λ60.017λ70.01λ80.0068λ90.0077λ100.0069λ110.023λ120.026λ130.011


### 4.2.3. Determining the Attribute Weights of the Wheat and Wheat Seed Data Set

The number of sample categories in the wheat seed data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, k is the number of attributes, and the meanings of each parameter are as given below. Table 9 shows the parameter values needed to calculate the weight of wheat seeds.Table 9
Main parameter explanation and value of the objective functions in determining wheat seed attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 170n2Samples with a category of 270n3Samples with a category of 370kNumber of data set attributes7The meanings of the other attributes are the same as in Table7. The vector values of the attributes of the center point under the three labels are sh=0.965,0.981,1.010,0.978,0.995,0.720,0.940T, sl=1.234,1.108,1.014,1.092,1.128,0.985,1.113T, and sm=0.799,0.909,0.975,0.929,0.875,1.294,0.946T. The rounded λkk=1,2…7 results are given below. Table 10 shows the calculated weights of wheat seed attributes.Table 10
Weight values of wheat seed characteristic attributes obtained by the accurate solution method.
WeightValueλ10.0328λ20.151λ30.504λ40.133λ50.071λ60.0012λ70.104


## 4.2.1. Determining the Attribute Weights of the Iris Data Set

The category number of the iris data set is 3, so the objective function used to determine the weights of the four attributes according to formula (1) is(5)sd=arg min∑i=1n1∑p=1kλp2rpi−sph2+∑j=1n2∑p=1kλp2rpj−spl2+∑r=1n3∑p=1kλp2rps−spm2,where n1,n2,n3 are the numbers of samples of mountain iris, chameleon iris, and Virginia iris, respectively; n is the total number of samples; k is the number of attributes; the iris has four attributes of calyx length, calyx width, petal length, and petal width (so k = 4); and the meanings of the other parameters are given below. Table 3 illustrates the number of irises and the k value for each category, and Table 4 shows the center vectors and parameter meanings of various types of flowers.Table 3
Main parameter explanation and the determined values of the objective functions of the iris attribute weights.
SymbolBrief explanationNumerical valuen1Number of mountain iris samples50n2Number of chameleon iris samples50n3Number of Virginia iris samples50kNumber of data set attributes4Table 4
Explanation of other parameters used in solving the objective function of iris attribute weights.
SymbolBrief explanationshCenter vector of mountain iris samplesslCenter vector of chameleon iris samplessmCenter vector of Virginia iris samplesxiSample attribute vector of the categoryWe separate the three categories of the data set and calculate the attribute vector values of the center points under the three tagssh=5.0063.4281.4620.246, sl=5.9362.774.261.326, and sm=6.5882.9745.5522.026. In the experiment, LINGO12.0 is used to solve, and the values are rounded to λkk=1,2,3,4 in Table 5.Table 5
The weight values of the iris characteristic attributes are accurately determined.
WeightValueλ10.053λ20.117λ30.107λ40.722


## 4.2.2. Determining the Wine Data Set Attribute Weights

The number of sample categories in the wine data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, and K is the number of attributes. The meaning of each parameter is given below. Table 6 lists the number and parameter significance of the three categories of the wine data set. Table 7 illustrates the three categories of wine center vector parameters.Table 6
Main parameters and values of the objective functions of wine attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 159n2Samples with a category of 271n3Samples with a category of 348kNumber of data set attributes13Table 7
Explanation of the other parameters of the objective functions of wine attribute weights.
SymbolBrief explanationshThe sample category has 1 center vectorslThe sample category has 2 center vectorssmThe sample category has 3 center vectorsxiThe sample attribute vector of the categoryThe vector values of the attributes of the center points under the three labels aresh=1.057,0.860,1.037,0.873,1.066,1.237,1.469,0.801,1.193,1.092,1.109,1.209,1.493T, sl=0.9444,0.827,0.948,1.038,0.947,0.984,1.025,1.004,1.024,0.610,1.103,1.066,0.695T, and sm=1.011,1.426,1.029,1.098,0.995,0.731,0.385,1.236,0.725,1.462,0.713,0.644,0.843T.The rounded resultsλ3λkk=1,2,…,13 are given below. Table 8 shows the weight values.Table 8
The weight values of the wine characteristic attributes are accurately obtained.
WeightValueλ10.745λ20.0043λ30.065λ40.036λ50.036λ60.017λ70.01λ80.0068λ90.0077λ100.0069λ110.023λ120.026λ130.011


## 4.2.3. Determining the Attribute Weights of the Wheat and Wheat Seed Data Set

The number of sample categories in the wheat seed data set is 3. The objective function is established according to formula (5). Data set is divided by the mean of the attributes for dimensionless processing, where n1,n2,n3 are the numbers of samples under different sample categories, n is the total number of samples, k is the number of attributes, and the meanings of each parameter are as given below. Table 9 shows the parameter values needed to calculate the weight of wheat seeds.Table 9
Main parameter explanation and value of the objective functions in determining wheat seed attribute weights.
SymbolBrief explanationNumerical valuen1Samples with a category of 170n2Samples with a category of 270n3Samples with a category of 370kNumber of data set attributes7The meanings of the other attributes are the same as in Table7. The vector values of the attributes of the center point under the three labels are sh=0.965,0.981,1.010,0.978,0.995,0.720,0.940T, sl=1.234,1.108,1.014,1.092,1.128,0.985,1.113T, and sm=0.799,0.909,0.975,0.929,0.875,1.294,0.946T. The rounded λkk=1,2…7 results are given below. Table 10 shows the calculated weights of wheat seed attributes.Table 10
Weight values of wheat seed characteristic attributes obtained by the accurate solution method.
WeightValueλ10.0328λ20.151λ30.504λ40.133λ50.071λ60.0012λ70.104


## 4.3. Analysis of the Experimental Results

The methods of K-means with accurately determined weights, traditional K-means, and K-means with entropy weights are used to cluster the iris, wine, and wheat seed data sets. The normalized mutual information and the confusion matrix [34] are used as evaluation criteria to evaluate the three methods.


### 4.3.1. Weight Entropy Method

The basic idea of the entropy weight method [35, 36] used to determine the objective weight is the index variability. Weight is determined according to the information entropy [37], which is the expectation of information content. The probability of the occurrence of a data value is negatively correlated with it. The higher the information entropy of an attribute is, the less information it can provide, the smaller the role it plays in evaluation, and the smaller its weight is. Table 11 shows the weight values of iris attributes obtained by the exact solution method. Table 12 shows the weight values of the attributes of the wine data set obtained by the exact solution method. Table 13 shows the weight values of the attributes of the wheat seed data set obtained by the exact solution method.Table 11
Weight values of iris attributes obtained by the entropy weight method.
WeightValueλ10.193λ20.112λ30.318λ40.376Table 12
Weight values of wine characteristic attributes obtained by the entropy weight method.
WeightValueλ10.049λ20.123λ30.022λ40.041λ50.059λ60.066λ70.109λ80.080λ90.067λ100.099λ110.069λ120.091λ130.120Table 13
Weight values of wheat seed characteristic attributes obtained by the entropy weight method.
WeightValueλ10.205λ20.158λ30.07λ40.155λ50.168λ60.115λ70.126


### 4.3.2. Iris Data Clustering Results

The experiment is implemented in the Python 3.8.5 environment, and the maximum number of K-means iterations after inputting the attribute weight is 200. The normalized mutual information is selected as the evaluation criterion, and the confusion matrix is established. The normalized mutual information can make the clustering results to 0-1 so that the clustering accuracy of the two methods can be seen intuitively [38]. The effect of clustering on a certain category can be obtained through a confusion matrix [39]. Clustering results can be visualized to make the results more intuitive [40]. The above methods are used to compare the results of the K-means algorithm with weights, K-means without weights, and K-means with weights determined by the entropy weight method. Table 14 shows the NMI of the iris data set after clustering by the three methods.Table 14
NMI of the iris data set after clustering by the three methods.
Normalized mutual information (NMI)K-means with the exact weight0.864K-means without weight0.758K-means with the entropy weight0.785NMI is an external evaluation standard method for clustering [41]. By calculating the normalized mutual information of the real labels and the labels after clustering, the accuracy of clustering can be seen [42, 43]. The NMI of the three methods after clustering the iris data set is shown in Table 14. First, it can be concluded from the table that the NMI after clustering by K-means with weights is approximately 0.11 higher than that after clustering without weights. The clustering effect of K-means after determining the attribute weights is better, which confirms the feasibility of this method. Second, when the results obtained by the entropy weight method are put into the K-means algorithm, the NMI after clustering is 0.785. It is 0.03 higher than that of traditional K-means without weights. However, the NMI after clustering of the algorithm proposed in this paper for accurately determining the weights is 0.08 higher than that of the entropy weight method. Finally, although the weight determined by the entropy weight method improves the accuracy of the iris data clustering class to a certain extent compared with clustering without weights, it is far from the improvement achieved by the weight determination method proposed in this paper. The confusion matrix after clustering is given below. Table 15 shows the confusion matrix of the effect of the three methods on iris clustering.Table 15
Confusion matrix of the three methods for clustering the iris data set.
Confusion matrixMountain irisChameleon irisVirginia irisAccurate methodReal category yMountain iris5000Chameleon iris0482Virginia iris0446Without weightMountain iris5000Chameleon iris0482Virginia iris01436Entropy weightsMountain iris5000Chameleon iris0491Virginia iris01337The confusion matrix is an effective tool for evaluating classifications and clustering criteria [44], as it can be used to clearly see in which categories the model does not perform well [45]. The confusion matrix [46, 47] after the three methods of clustering is shown in Table 15. First, it can be seen from the table that the clustering effect of the three methods is equally good for the mountain iris. These samples can be clustered accurately. All three methods are largely accurate in the category of the chameleon iris, but there is a large difference among the three in the category of the Virginia iris. K-means without weights incorrectly clustered 14 samples of Virginia iris into the category of chameleon iris. Compared with K-means clustering without weights, the improvement of K-means clustering after weight determination by the entropy weight method is not very large. Second, for the clustering of Virginia iris, the weight clustering results are almost the same as those of all attributes after weight determination by the entropy weight method. After the weights are determined by the entropy weight method, 13 Virginia irises are incorrectly clustered into the chameleon iris category, while only 14 samples are incorrectly classified even with uncertain weights. Neither method could accurately cluster Virginia irises, and it was more difficult to cluster Virginia irises than the other two iris categories. Figure 2 shows that the calyx and petal lengths and widths of the Virginia iris and chameleon iris are similar. The data are mixed and difficult to distinguish, which means that the two methods cannot distinguish the two flower categories well. The difference in the properties of the mountain iris and the other two flowers is relatively large. The weight obtained by the algorithm with the accurate solution is applied to K-means, which can distinguish the two categories well, proving the accuracy and efficiency of the method. Finally, the effect of K-means clustering determined by the entropy weight method is visualized.Figure6 shows the results of clustering the iris data set with the weights obtained by our method. Figure 7 shows the clustering results without attribute weights. Figure 8 shows the clustering results of the weights determined by the entropy weight method. The effect diagram after clustering shows more intuitively that some sample points are still mixed in the clustering results of chameleon iris and Virginia iris by K-means without weights. These points are not effectively divided into different clusters. However, K-means with accurately determined weights has a better effect on the clustering of the two types. Points of different categories are effectively clustered into different clusters.Figure 6
K-means clustering effect diagram of the iris data after the exact solution method is used to determine the weights.Figure 7
Effect diagram of K-means without weights on iris data clustering.Figure 8
K-means clustering effect diagram of iris data after the entropy weight method is used to determine the weights.


### 4.3.3. Wine Data Clustering Results

The wine data set has more attributes than the iris data set. The results of the following three methods are compared: the K-means algorithm for calculating the weights by the exact solution method, K-means without weights, and K-means with weights determined by the entropy method. Table16 shows the NMI values of the three methods for clustering the wine data set, and Table 17 shows the confusion matrix of the three methods for clustering the wine data set.Table 16
NMI aggregated by the three methods for wine data.
Normalized mutual information (NMI)K-means with the exact weight0.865K-means without weights0.765K-means with entropy weight0.765Table 17
Confusion matrix of the three methods for clustering the wine data set.
Confusion matrixCategory y_pred after K-means clustering with the exact solution methodCategory 1Category 2Category 3Accurate methodReal category yCategory 15900Category 25642Category 30048Without weightCategory 15801Category 22609Category 31047Entropy weightsCategory 15810Category 22609Category 30147According to the NMI after clustering by the three methods, the method for solving the weight proposed in this paper improves the results by approximately 0.1 compared with those of the other two methods. The entropy weight method does not improve the results much in the wine data clustering class, so different weight solving methods apply to different situations. According to the confusion matrix after clustering by the three methods, the exact solution method performs better than the other two methods on the three sample categories. There is little difference between the entropy weight method and K-means without weights. Figure9 shows the clustering results of the wine data set by the method in this paper, Figure 10 shows the clustering results without attribute weights, and Figure 11 shows the entropy weighting method clustering results.Figure 9
Effect diagram of K-means clustering of the wine data after the exact solution method is used to determine the weights.Figure 10
Effect diagram of K-means without weights on wine data clustering.Figure 11
Effect diagram of K-means clustering of the wine data after the entropy weight method is used to determine the weights.


### 4.3.4. Cluster Results on Wheat Seed Data

The number of attributes in the wheat seed data set is between those of the iris data set and the wine data set. The results of the weighted K-means algorithm, the K-means package in SKLearn, and weighted K-means with weights determined by the entropy weight method are compared below. Table18 shows the NMI results of the three methods for clustering wheat seeds, and Table 19 shows the confusion matrix results after clustering.Table 18
NMI of wheat seed data aggregated by the three methods after classification.
Normalized mutual information (NMI)K-means with the exact weight0.673K-means without weights0.524K-means with entropy weight0.621Table 19
Confusion matrix of the three methods for clustering the wheat seed data set.
Confusion matrixCategory 1Category 2Category 3Accurate methodReal category yCategory 155213Category 29610Category 31069Without weightCategory 158111Category 210600Category 320050Entropy weightsCategory 160100Category 212579Category 33265The attribute importance of the wheat seed data set varies. Compared with the K-means clustering results without weights, the normalized mutual information after K-means clustering with weights is greatly improved. The normalized mutual information after applying the exact solution method and the entropy weight method of determining the weights is improved by 0.15 and 0.1, respectively. However, compared with the entropy weight method, the weight method proposed in this paper improves the normalized mutual information by 0.05, and the clustering effect is better.It can be seen from the confusion matrix after clustering by the three methods that the improvement of weighted K-means compared with unweighted K-means is mainly in the data set of categories 3. The number of correct samples in the clustering of the precise solution method and entropy weight method is increased by 19 and 15, respectively, compared with that of traditional K-means. Compared with the entropy weight method, the exact solution method performs better in the clustering of category 3. Figure12 represents the clustering result of wheat seed data by the method in this paper, while Figure 13 shows the clustering results of the unweighted data set. Figure 14 shows the clustering results of the wheat seed data set by the entropy weight method. As seen from the visualization, the clustering effects of the accurate solution method and entropy weight method are significantly better than that of traditional K-means. Samples of different categories are divided into different clusters.Figure 12
K-means clustering effect diagram of wheat seed data after the exact solution method is used to determine the weights.Figure 13
Effect diagram of K-means without weights on wheat seed data clustering.Figure 14
The K-means clustering effect of wheat seed data after the entropy weight method is used to determine the weights.


## 4.3.1. Weight Entropy Method

The basic idea of the entropy weight method [35, 36] used to determine the objective weight is the index variability. Weight is determined according to the information entropy [37], which is the expectation of information content. The probability of the occurrence of a data value is negatively correlated with it. The higher the information entropy of an attribute is, the less information it can provide, the smaller the role it plays in evaluation, and the smaller its weight is. Table 11 shows the weight values of iris attributes obtained by the exact solution method. Table 12 shows the weight values of the attributes of the wine data set obtained by the exact solution method. Table 13 shows the weight values of the attributes of the wheat seed data set obtained by the exact solution method.Table 11
Weight values of iris attributes obtained by the entropy weight method.
WeightValueλ10.193λ20.112λ30.318λ40.376Table 12
Weight values of wine characteristic attributes obtained by the entropy weight method.
WeightValueλ10.049λ20.123λ30.022λ40.041λ50.059λ60.066λ70.109λ80.080λ90.067λ100.099λ110.069λ120.091λ130.120Table 13
Weight values of wheat seed characteristic attributes obtained by the entropy weight method.
WeightValueλ10.205λ20.158λ30.07λ40.155λ50.168λ60.115λ70.126


## 4.3.2. Iris Data Clustering Results

The experiment is implemented in the Python 3.8.5 environment, and the maximum number of K-means iterations after inputting the attribute weight is 200. The normalized mutual information is selected as the evaluation criterion, and the confusion matrix is established. The normalized mutual information can make the clustering results to 0-1 so that the clustering accuracy of the two methods can be seen intuitively [38]. The effect of clustering on a certain category can be obtained through a confusion matrix [39]. Clustering results can be visualized to make the results more intuitive [40]. The above methods are used to compare the results of the K-means algorithm with weights, K-means without weights, and K-means with weights determined by the entropy weight method. Table 14 shows the NMI of the iris data set after clustering by the three methods.Table 14
NMI of the iris data set after clustering by the three methods.
Normalized mutual information (NMI)K-means with the exact weight0.864K-means without weight0.758K-means with the entropy weight0.785NMI is an external evaluation standard method for clustering [41]. By calculating the normalized mutual information of the real labels and the labels after clustering, the accuracy of clustering can be seen [42, 43]. The NMI of the three methods after clustering the iris data set is shown in Table 14. First, it can be concluded from the table that the NMI after clustering by K-means with weights is approximately 0.11 higher than that after clustering without weights. The clustering effect of K-means after determining the attribute weights is better, which confirms the feasibility of this method. Second, when the results obtained by the entropy weight method are put into the K-means algorithm, the NMI after clustering is 0.785. It is 0.03 higher than that of traditional K-means without weights. However, the NMI after clustering of the algorithm proposed in this paper for accurately determining the weights is 0.08 higher than that of the entropy weight method. Finally, although the weight determined by the entropy weight method improves the accuracy of the iris data clustering class to a certain extent compared with clustering without weights, it is far from the improvement achieved by the weight determination method proposed in this paper. The confusion matrix after clustering is given below. Table 15 shows the confusion matrix of the effect of the three methods on iris clustering.Table 15
Confusion matrix of the three methods for clustering the iris data set.
Confusion matrixMountain irisChameleon irisVirginia irisAccurate methodReal category yMountain iris5000Chameleon iris0482Virginia iris0446Without weightMountain iris5000Chameleon iris0482Virginia iris01436Entropy weightsMountain iris5000Chameleon iris0491Virginia iris01337The confusion matrix is an effective tool for evaluating classifications and clustering criteria [44], as it can be used to clearly see in which categories the model does not perform well [45]. The confusion matrix [46, 47] after the three methods of clustering is shown in Table 15. First, it can be seen from the table that the clustering effect of the three methods is equally good for the mountain iris. These samples can be clustered accurately. All three methods are largely accurate in the category of the chameleon iris, but there is a large difference among the three in the category of the Virginia iris. K-means without weights incorrectly clustered 14 samples of Virginia iris into the category of chameleon iris. Compared with K-means clustering without weights, the improvement of K-means clustering after weight determination by the entropy weight method is not very large. Second, for the clustering of Virginia iris, the weight clustering results are almost the same as those of all attributes after weight determination by the entropy weight method. After the weights are determined by the entropy weight method, 13 Virginia irises are incorrectly clustered into the chameleon iris category, while only 14 samples are incorrectly classified even with uncertain weights. Neither method could accurately cluster Virginia irises, and it was more difficult to cluster Virginia irises than the other two iris categories. Figure 2 shows that the calyx and petal lengths and widths of the Virginia iris and chameleon iris are similar. The data are mixed and difficult to distinguish, which means that the two methods cannot distinguish the two flower categories well. The difference in the properties of the mountain iris and the other two flowers is relatively large. The weight obtained by the algorithm with the accurate solution is applied to K-means, which can distinguish the two categories well, proving the accuracy and efficiency of the method. Finally, the effect of K-means clustering determined by the entropy weight method is visualized.Figure6 shows the results of clustering the iris data set with the weights obtained by our method. Figure 7 shows the clustering results without attribute weights. Figure 8 shows the clustering results of the weights determined by the entropy weight method. The effect diagram after clustering shows more intuitively that some sample points are still mixed in the clustering results of chameleon iris and Virginia iris by K-means without weights. These points are not effectively divided into different clusters. However, K-means with accurately determined weights has a better effect on the clustering of the two types. Points of different categories are effectively clustered into different clusters.Figure 6
K-means clustering effect diagram of the iris data after the exact solution method is used to determine the weights.Figure 7
Effect diagram of K-means without weights on iris data clustering.Figure 8
K-means clustering effect diagram of iris data after the entropy weight method is used to determine the weights.


## 4.3.3. Wine Data Clustering Results

The wine data set has more attributes than the iris data set. The results of the following three methods are compared: the K-means algorithm for calculating the weights by the exact solution method, K-means without weights, and K-means with weights determined by the entropy method. Table16 shows the NMI values of the three methods for clustering the wine data set, and Table 17 shows the confusion matrix of the three methods for clustering the wine data set.Table 16
NMI aggregated by the three methods for wine data.
Normalized mutual information (NMI)K-means with the exact weight0.865K-means without weights0.765K-means with entropy weight0.765Table 17
Confusion matrix of the three methods for clustering the wine data set.
Confusion matrixCategory y_pred after K-means clustering with the exact solution methodCategory 1Category 2Category 3Accurate methodReal category yCategory 15900Category 25642Category 30048Without weightCategory 15801Category 22609Category 31047Entropy weightsCategory 15810Category 22609Category 30147According to the NMI after clustering by the three methods, the method for solving the weight proposed in this paper improves the results by approximately 0.1 compared with those of the other two methods. The entropy weight method does not improve the results much in the wine data clustering class, so different weight solving methods apply to different situations. According to the confusion matrix after clustering by the three methods, the exact solution method performs better than the other two methods on the three sample categories. There is little difference between the entropy weight method and K-means without weights. Figure9 shows the clustering results of the wine data set by the method in this paper, Figure 10 shows the clustering results without attribute weights, and Figure 11 shows the entropy weighting method clustering results.Figure 9
Effect diagram of K-means clustering of the wine data after the exact solution method is used to determine the weights.Figure 10
Effect diagram of K-means without weights on wine data clustering.Figure 11
Effect diagram of K-means clustering of the wine data after the entropy weight method is used to determine the weights.


## 4.3.4. Cluster Results on Wheat Seed Data

The number of attributes in the wheat seed data set is between those of the iris data set and the wine data set. The results of the weighted K-means algorithm, the K-means package in SKLearn, and weighted K-means with weights determined by the entropy weight method are compared below. Table18 shows the NMI results of the three methods for clustering wheat seeds, and Table 19 shows the confusion matrix results after clustering.Table 18
NMI of wheat seed data aggregated by the three methods after classification.
Normalized mutual information (NMI)K-means with the exact weight0.673K-means without weights0.524K-means with entropy weight0.621Table 19
Confusion matrix of the three methods for clustering the wheat seed data set.
Confusion matrixCategory 1Category 2Category 3Accurate methodReal category yCategory 155213Category 29610Category 31069Without weightCategory 158111Category 210600Category 320050Entropy weightsCategory 160100Category 212579Category 33265The attribute importance of the wheat seed data set varies. Compared with the K-means clustering results without weights, the normalized mutual information after K-means clustering with weights is greatly improved. The normalized mutual information after applying the exact solution method and the entropy weight method of determining the weights is improved by 0.15 and 0.1, respectively. However, compared with the entropy weight method, the weight method proposed in this paper improves the normalized mutual information by 0.05, and the clustering effect is better.It can be seen from the confusion matrix after clustering by the three methods that the improvement of weighted K-means compared with unweighted K-means is mainly in the data set of categories 3. The number of correct samples in the clustering of the precise solution method and entropy weight method is increased by 19 and 15, respectively, compared with that of traditional K-means. Compared with the entropy weight method, the exact solution method performs better in the clustering of category 3. Figure12 represents the clustering result of wheat seed data by the method in this paper, while Figure 13 shows the clustering results of the unweighted data set. Figure 14 shows the clustering results of the wheat seed data set by the entropy weight method. As seen from the visualization, the clustering effects of the accurate solution method and entropy weight method are significantly better than that of traditional K-means. Samples of different categories are divided into different clusters.Figure 12
K-means clustering effect diagram of wheat seed data after the exact solution method is used to determine the weights.Figure 13
Effect diagram of K-means without weights on wheat seed data clustering.Figure 14
The K-means clustering effect of wheat seed data after the entropy weight method is used to determine the weights.


## 5. Discussion and Conclusions

Class distance-based data classification algorithms are used to deal with different scenarios, where determining weights is an important and difficult problem. Based on the data value itself, this paper proposes a precisely determined distance weight, which makes the method more objective. The weight is determined only by solving the minimum function, and methods such as the entropy weight method and principal component analysis (PCA) are not needed. After determining the minimum Euclidean distance between the attribute vector of each category and the center point vector of the category to determine the weight, the obtained result is applied to the K-means clustering algorithm. Experiments were conducted using normalized mutual information as an evaluation criterion and a confusion matrix to evaluate clustering details. In this paper, we cluster the iris data set, wine data set, and wheat seed data set. The results show that, using the weight determination method proposed in this paper, confusion matrix and normalized mutual information results are better than the other two methods. Based on entropy and traditional K-means, the solution method is proven to be effective. Finally, the method is compared with the entropy weight method, to compare clustering results. The effect of the entropy weight method in determining class weight is not as good as that of the method proposed in this paper, which proves the accuracy and efficiency of this method. However, this paper only uses Euclidean distance as a distance function to measure each sample point and the center point to which it belongs. There are other distance functions in addition to Euclidean distance. Validating this approach with other distance functions is our next step. In addition, three classic machine learning data sets are taken as examples to demonstrate the effectiveness and efficiency of this method for determining weights. However, different weight determination methods are suitable for different data sets, and more verification is required for different scenarios and different data sets. For other methods, such as neural networks, further verification is required in future work. The distance-based weight determination method proposed in this paper still needs to be improved in the future, but the distance-based weight determination method is different from the subjective, entropy, and variance methods and provides a new idea for future weight determination.



---

*Source: 6936335-2022-05-27.xml*