bixo/bixo — pull request #49 by michelecos (2012-03-29 10:11:50+00:00)
Title: Support for a site list

I have added support for a site list to fetch in SimpleCrawlTool.
There is a new option for the site list, a function that uses UrlImporter to populate the initial crawldb and the URL filter that limits to the main site is disabiled if it is a multisite crawl.