Cleanup, functions on crawl directive
Client controller for crawls debugging
Fix merge error