A comparison of experiments with bisecting-spherical K-means clustering and SVD algorithms

Several experiments comparing clustering and SVD techniques can be found in the literature. In this paper we propose a modified version of the Spherical k -means clustering algorithm, the BisectingSpherical k -means clustering algorithm. The bisecting clustering algorithm is used to determine the initial set in the Spherical k-means clustering algorithm. We have set up a set of experiments to compare the SVD with different number of singular values in order to find an optimal solution. Analogously we have done with our modified version of the Spherical k -means clustering algorithm, with different number of clusters. Finally we have compared both techniques with respect to precision-recall ratios, concluding that both techniques perform similarly in terms of precision-recall ratios. 1. BACKGROUND AND MOTIVATION The information retrieval (IR) models used in our experiments are classified within the classic model, precisely on the vector space model. The concrete models used are the generalized vector space model and the latent semantic indexing (LSI). The basic vector space model is defined with a matrix of frequencies whereas in the generalized vector model the matrix of frequencies is substituted by a matrix of weights, both generically known as a matrix of terms by documents. The paper started with this section, background and motivation where we briefly explain the organization of the paper and depict the series of experiments and comparisons performed (see fig.1). Next follows the preprocessing section where we have thoroughly explained the stemming algorithm and the reduction heuristics used for reducing the matrix space. Additionally it is explained the IR validation technique used along this paper for the whole set of experiments, known as average precision-recall ratio. In section 3 we present an overview of the SVD technique applied to IR whereas in section 4 it is presented the modified algorithm version of the Spherical k-means clustering algorithm. In section 5 we start by explaining the data set used and present a series of interesting statistics. We compare the optimal reduced frequency and weight matrices. Next we compare several singular values within the SVD. After choosing the optimal SVD approximation we compare it with the optimal weight matrix. Analogously we chose the optimal number of clusters within the bisectingspherical k-means clustering algorithm. Finally we compare the optimal SVD and the optimal bisecting-spherical k-means clustering algorithms. We complete the paper with conclusions and future work. 2. PREPROCESSING Several stemming algorithms were considered to pre-process our target collection. In order to evaluate which stemming algorithm to choose, we downloaded a set of stemming algorithms from the Efficient Stemmer Generation Project [Fox & Fox 2001]. This site includes several stemming implementations and a corpus of words to be tested. We have slightly modified the Paice stemming algorithm to deal with hyphenated words. As stated in [Paice 1994] the Porter algorithm is classified as a light stemming algorithm prone to under-stemming errors and the Paice and Lovins as heavy stemming algorithms prone to over-stemming errors. Light and heavy stemming algorithms cannot be compared in terms of accuracy as each type is suited for a different task. On the other hand in terms of performance accuracy we decided to include the Paice algorithm in our experiments as a better stemming algorithm as indicated in [Paice 1994]. In table 1 we can see the reduction rate obtained by the different stemmers. Figure 1. Steps followed in our experiments. Porter Lovins Paice No Input terms 49528 49656 49656 No. output terms 41283 35029 33297 Reduction 16.65% 29.45% 32.94% Table 1. Comparison of Stemming Algorithms In the preprocessing we have also included a set of simple heuristics to eliminate terms that only appear in a small documents set (or in a certain percentage). The following table presents the different chosen options: No. Docs. % Docs Number Terms Size Reduction Number Non Zeros 0 0 % 12476 0 % 81082 1 0.24 % 6545 47.54 % 75151 2 0.47 % 4803 61.50 % 71667 3 0.71 % 3903 68.72 % 68967 4 0.94 % 3321 73.38 % 66639 5 1.18 % 2913 76.65 % 64599 Table 2. Heuristics used for pruning the term collection. The criteria used to compare the reduction heuristics, has been the average precision-recall ratio as mentioned in [Baeza 1999].