refactor the crawler