Add crawl delay time

* Add option for config crawl delay and threads for main crawler list