Refactor crawler code