html2words should return list of str not str
added function that determines whether a urls should be collected or not
able to collect urls from webpage for expanding purpose
moved read_classifier to a separate file to avoid the notorious pickling error, no such attribute, NBClassifier
removed unused libs sklearn numpy, scipy from requirements.txt
added label argument for classifier related tests
fixed the shell script replaces tabs to spaces bug
added the scrapy-based spider
add more strict url filtering (allowing only http and https scheme)
baseline crawler able to crawl and save results, remvoved debugging info in classifier.py
a late addition of a test html page