take the changes for all Crawlers
fix some tests
update from the master branch
Merge pull request #191 from mimmi20/issue-105-bots

take the changes for all Crawlers