A Hardware Implementation of Hierarchical Clustering for Text Documents

Clustering is a form of unsupervised learning where a collection of unlabeled data points is partitioned into groups (clusters) according to some similarity metric, maximizing similarity intracluster while minimizing similarity intercluster. In hierarchical clustering, the groups are clustered into subgroups, etc., organizing the data as a tree. Many clustering algorithms are superlinear, while pertinent datasets are growing in size exponentially. Since clustering algorithms also tend to be highly parallelizable, there is a strong motivation for speedup via hardware acceleration. We present a simple hierarchical clustering algorithm for text documents, adapt it for hardware acceleration, and implement on the FPX platform. To our knowledge, this is the first work to present a hierarchical clustering algorithm designed for straightforward hardware implementation, and ours is the first hardwareaccelerated implementation of a hierarchical clustering algorithm. I. MOTIVATION AND OVERVIEW As databases of interest grow exponentially larger, data-mining algorithms are struggling to keep up (cf. [10], [4]). This is an broad phenomenon, affecting at least the following areas: • Gene sequences and other bioinformatics data sources • Sales and other business transactions • The www • Various data sources utilized within the intelligence community Thus, there is compelling motivation to move data-mining algorithms into hardware in order to accelerate them (i.e., take advantage of specialized circuitry). In this paper, we focus on a particular data-mining problem, the hierarchical clustering of text documents based on document similarity. To our knowledge, this is the first work to present a hierarchical clustering algorithm designed for straightforward hardware implementation. 1 Section II briefly reviews the clustering problem in general, with a particular focus on the clustering of text documents. Section III describes our clustering algorithm. The FPX platform and LEON2 Processor, with we have chosen for our implementation, are introduced in Section IV, and their selection justified. In Section VI we then describe how the primary computation of the algorithm (scoring) may be offloaded onto a specialized circuit, describe the speedup in Section VII), and conclude in Section VIII. II. THE CLUSTERING PROBLEM The clustering problem is to partition a given a set of items into subsets, typically maximizing intracluster similarity while minimizing intercluster similarity, according to a given similarity metric. In hierarchical clustering, the clusters are themselves recursively clustered. Well-known “flat” (as opposed to hierarchical) clustering algorithms include kmeans and its generalization, expectation maximization [7]. A well-studied special case is where the items correspond to text documents and the similarity metric is based on viewing a document as a “bagof-words”, or “bag-of-word-disjunctions” [16], [17]. That is, documents are points in a high-dimensional space where each dimension corresponds to the frequency/occurrence of one or more words in the document. There are a number of important factors to consider in crafting an approach to text document clustering which have been identified in the literature: A number of versions of the flat (non-hierarchical) k-means algorithm have been implemented in hardware, most frequently for image clustering (cf. [5]). • The similarity measure used is as important as the algorithm chosen; normalized measures such as the cosine measure,