Separated logic and usage of crawler in two files.