Clustering Procedure in SAS Lei Department of Electrical and Computer Engineering University of Victoria, Canada Introduction The clustering is to group data into different groups by computing the distance between the datasets. The distance between the datasets is defined the closeness of the datasets. In machine learning, the clustering is also called unsupervised learning, in which the datasets used do not have labeled classes or groups. The methods applied to compute the distance vary from some simple measures to very complicate ones depending on specific research purposes and the only difference is how to compute the distance between two clusters. The general process of the clustering is that each observation or data record begins in a cluster by itself; then, the two closest clusters are merged to form a new one to replace the two old ones; continuing to repeat the merging the two closest clusters until only one cluster is left. The CLUSTER Procedure in SAS hierarchically clusters the data records or observations in a SAS data set by using one of the eleven methods. The data can be coordinates or distances. If the data re coordinates, PROC CLUSTER computes (possibly squared) Euclidean distances. For coordinate data, Euclidean distances are computed from differences between coordinate values as the use of differences has several important consequences: For differences to be valid, the variables must have an interval or stronger scale of measurements. Ordinal or ranked data are generally not appropriate for cluster analysis. For Euclidean distances to be comparable, equal differences should have equal practical importance. Variables with large variances tend to have more effect on the resulting clusters than variables with small variances. The eleven clustering methods supported in SAS are: average linkage, the centroid method, complete linkage, density linkage (including Wong’s hybrid and kth-neighbor methods), maximum likelihood for mixtures of spherical multivariate normal distributions with equal variances but possibly unequal mixing proportions, the flexible-beta method, McQuitty’s similarity analysis, the median method, single linkage, density linkage, and Ward’s
METHOD=WARD if the data set contains both of the following: a variable giving the number of original observations in each cluster. a variable giving the root mean squared standard deviation of each other. VAR statement lists numeric variables to be used in the cluster analysis. If it is omitted, all numeric variables not listed in other statements are used. Clustering Methods There are eleven clustering methods provided in SAS. The method of average linkage (METHOD = AVERAGE or AVG) computes the average distance between pairs of observations, one in each other. Average linkage tends to join clusters with small variances, and it is slightly biased toward producing clusters with the same variance. In the centroid method (METHOD = CENTROID), the distance between two clusters is defined as the (squared) Euclidean distance between their centroids or means. This method is more robust to outliers than most other hierarchical methods but in other respects might not perform as well as Ward’s method or average linkage. Complete linkage method (METHOD = COMPLETE) use the maximum distance between an observation in one cluster and an observation in the cluster as the distance between two clusters. It is strongly biased toward producing clusters with roughly equal diameters, and it can be severely distorted by moderate outliers. Density linkage (METHOD = DENSITY) is referred to a class of clustering methods that use nonparametric probability density estimates. It consists of two steps: a new dissimilarity measure based on density estimates and adjacencies is computed. a single linkage cluster analysis is performed. There are three types of density linkage: the kth-nearest-neighbor method, the uniform-kernel method and Wong’s hybrid method. EML method (METHOD = EML) clusters to maximize the likelihood at each level of the hierarchy under the following assumptions: multivariate normal mixture equal spherical covariance matrices unequal sampling probabilities This method is similar to Ward’s minimum-variance method but removes the bias toward clusters. The distance computed by the method (METHOD = FLEXIBLE) is presented as