Implements crawler logic in a separated module 'crawler.engine'.