Removed some dependencies and unneeded methods for the crawling 'framework'. They should be added in some form of a utils package