Initial commit of cluster code.
Bug fix - iterations not being respected. Adding satalite data to document class.
Fixed top-term reporting. Implemented Chinese Restaurant Process for cluster cardinality picking. Some interface enhancements on the cluster_blog_feed.rb script.
Changed algorithm to use a sparse vector implementation (a fancy hash). Run times are excellent, but validation needs doing. Also, a stopword list and filter list are deperately needed to give more meaning to the clusters bing produced.
Adding stemming library. This should reduce the number of terms. Next up is filtering HTML and other markup.
Refactoring filter into a pipeline model and into its own file.
Bug fixes. Adding a test to exercise the DocumentFilter object.
Increasing stopword list using brown corpus II.
Adding stemming and HTML tag stripping using nokogiri >= 1.4.1. Some filter fixes are also in this checkin.
Some code cleanup and adding a missing stem file.
Adding license.
Interim commit. Fixed use of sparse vectors. Validation of clustering coming.
Adding licensing information.
Filter enhancements. Trim whitespace. Split more sentences.
More intelligent n-gram building. Term limit feature to only keep term_limit most relevant terms based on (tf-idf).
Randomize which cluster is set as the intial minimum. This randomization prevents a dominant "sink" cluster from forming. Also, switching + to * operator which was creating incorrect cluster centers.
Code touch-up. Call to store returns the stored value per Hash definition.
Updates to make unit tests work with new SparseVector type and some minor features/options for cluster_blog_feed.rb
Refactor to accomodate new algorithms in sclust.
Refactoring sclust to sclust/kmean.