Added the option 'ignoreRelative', added more documentation, re-factoring of how options are configured
Added examples how crawling results can be filtered and the same crawler can be reused for several pages
Ignoring hashes when crawling in order not to crawl the same page twice when there are links to different fragments inside that pages
Added the ability to specify which URLs should be crawled