Moved all functions to crawlerUtil