Continued refactoring the crawler
Made default log level "notice" instead of "debug"
Added validation to the configuration
Refactored the crawler