improved crawl script, bug fixes, algorithm tuning, optimization. it produces decent output now provided it has sufficient (500MB+) input