Adding ignore and timeout options.
Implementing time limit while crawling pages.
Implementing ignore filter patters, improving filters, fixing some
validations and cleaning up a bunch of things.